Traces - Request Journeys

Understanding traces and how they represent complete request journeys through your AI application

A trace represents the complete journey of a request through your AI application, from the initial input to the final output. Think of it as a story that shows how your application processes a single user request.

🎯 What is a Trace?

A trace is a distributed operation that can span multiple services, functions, and external API calls. In AI applications, a trace typically represents:

A single user query through your chatbot
A complete RAG pipeline from question to answer
An agent workflow with multiple steps and decisions
A batch processing job with multiple AI operations

🏗️ Trace Structure

Every trace contains:

Trace ID: Unique identifier for the entire request
Root Span: The main operation that started the trace
Child Spans: Sub-operations within the main operation
Attributes: Key-value pairs with metadata
Events: Point-in-time occurrences during execution
Status: Success, error, or other completion state

📊 Visual Timeline

Here's how a trace looks in the Noveum dashboard:

🔄 Trace Lifecycle

1. Trace Creation

A trace is created when a new request starts:

from noveum_trace import trace_operation
 
# This creates a new trace
with trace_operation("customer-support-query") as span:
    # Your application logic here
    pass

2. Span Addition

Spans are added as the request progresses:

with trace_operation("customer-support-query") as main_span:
    # Add customer context
    main_span.set_attributes({
        "customer.id": "cust_123",
        "query.type": "technical_support"
    })
    
    # Add a child span for LLM call
    with trace_llm(model="gpt-4", provider="openai") as llm_span:
        # LLM operation
        pass

3. Trace Completion

The trace is automatically completed when the root span ends:

with trace_operation("customer-support-query") as span:
    try:
        # Process the request
        result = process_customer_query()
        span.set_status("success")
        return result
    except Exception as e:
        span.set_status("error", str(e))
        raise

🎯 Trace Patterns in AI Applications

Simple LLM Call

# Single LLM operation
with trace_llm(model="gpt-4", provider="openai") as span:
    response = openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": "Hello"}]
    )

RAG Pipeline

# Multi-step RAG process
with trace_operation("rag-pipeline") as main_span:
    # Step 1: Generate embeddings
    with trace_operation("generate-embeddings") as emb_span:
        embeddings = generate_embeddings(query)
    
    # Step 2: Retrieve documents
    with trace_operation("retrieve-documents") as ret_span:
        documents = vector_search(embeddings)
    
    # Step 3: Generate answer
    with trace_llm(model="gpt-4", provider="openai") as llm_span:
        answer = generate_answer(query, documents)

Multi-Agent Workflow

# Agent coordination
with trace_operation("multi-agent-workflow") as main_span:
    # Agent 1: Research
    with trace_agent(agent_type="researcher") as research_span:
        research_data = research_agent.analyze(topic)
    
    # Agent 2: Writing
    with trace_agent(agent_type="writer") as writer_span:
        report = writer_agent.create_report(research_data)
    
    # Agent 3: Review
    with trace_agent(agent_type="reviewer") as review_span:
        final_report = reviewer_agent.review(report)

📈 Trace Attributes

Traces can contain rich metadata through attributes:

System Attributes

span.set_attributes({
    "trace.id": "trace_12345",
    "trace.duration_ms": 2300,
    "trace.status": "success",
    "trace.start_time": "2024-01-15T10:30:00Z"
})

Business Attributes

span.set_attributes({
    "customer.id": "cust_12345",
    "customer.tier": "premium",
    "query.category": "technical_support",
    "query.priority": "high",
    "query.language": "en"
})

AI-Specific Attributes

span.set_attributes({
    "ai.model": "gpt-4",
    "ai.provider": "openai",
    "ai.temperature": 0.7,
    "ai.max_tokens": 1000,
    "ai.cost_usd": 0.0023
})

🎪 Trace Events

Events represent point-in-time occurrences during trace execution:

Business Events

# Customer interaction events
span.add_event("customer.query.received", {
    "timestamp": "2024-01-15T10:30:00Z",
    "query.length": 45,
    "query.sentiment": "neutral"
})
 
span.add_event("customer.query.processed", {
    "timestamp": "2024-01-15T10:30:02Z",
    "processing_time_ms": 2000,
    "confidence_score": 0.85
})

AI Events

# Model decision events
span.add_event("ai.model.selected", {
    "model": "gpt-4",
    "reason": "complex_query",
    "fallback_used": False
})
 
span.add_event("ai.response.generated", {
    "tokens_used": 150,
    "finish_reason": "stop",
    "response_time_ms": 1800
})

🔍 Trace Analysis

Performance Analysis

Duration: Total time from start to finish
Latency: Time spent in each operation
Bottlenecks: Slowest operations in the trace
Throughput: Requests processed per second

Cost Analysis

Token Usage: Input and output tokens
API Costs: Cost per provider and model
Total Cost: End-to-end request cost
Cost Attribution: Which operations drive costs

Quality Analysis

Success Rate: Percentage of successful requests
Error Patterns: Common failure points
Response Quality: AI output quality metrics
User Satisfaction: Business quality indicators

🚀 Next Steps

Now that you understand traces, explore these related concepts:

Spans - Individual operations within traces
Attributes - Metadata and context
Events - Point-in-time occurrences

Best Practices

Traces Best Practices - Learn how to create effective traces

Traces are the foundation of observability in AI applications. They provide the complete picture of how your application processes requests, making it easy to understand, debug, and optimize your AI workflows.