Noveum.ai
Noveum Docs
IntegrationsLangGraph IntegrationIterative Research Agent

Iterative Research Agent

Learn how to trace iterative research agents with self-loops using Noveum Trace

This guide shows you how to trace iterative research agents that can loop back to refine their work. You'll learn how to monitor self-loops, state evolution, and iterative refinement processes.

🎯 Use Case

Research Assistant Agent: An agent that conducts research on a topic, evaluates the quality of information gathered, and can loop back to gather more information if needed. We'll trace the complete iterative process.

🚀 Complete Working Example

Here's a complete, working example based on langgraph_agent_example.py:

import os
from typing import Annotated, Literal, TypedDict
from dotenv import load_dotenv
import noveum_trace
from noveum_trace import NoveumTraceCallbackHandler
from langchain_core.messages import AIMessage, HumanMessage
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langgraph.graph import END, StateGraph

load_dotenv()

# Initialize Noveum Trace
noveum_trace.init(
    api_key=os.getenv("NOVEUM_API_KEY"),
    project="customer-support-bot",
    environment="development"
)

# Define the research state
class ResearchState(TypedDict):
    messages: Annotated[list, "The messages in the conversation"]
    research_topic: str
    research_notes: Annotated[list, "Research notes gathered"]
    evaluation_score: float
    max_iterations: int
    current_iteration: int
    research_complete: bool

# Define research tools
@tool
def search_web(query: str) -> str:
    """Search the web for information about a query."""
    # Simulate web search with realistic results
    search_results = {
        "artificial intelligence": "AI is a branch of computer science focused on creating intelligent machines...",
        "machine learning": "Machine learning is a subset of AI that enables computers to learn without explicit programming...",
        "deep learning": "Deep learning uses neural networks with multiple layers to process data...",
        "natural language processing": "NLP is a field of AI that focuses on the interaction between computers and human language..."
    }
    
    # Return relevant results based on query
    for key, value in search_results.items():
        if key in query.lower():
            return f"Search results for '{query}': {value}"
    
    return f"Search results for '{query}': General information about the topic."

@tool
def analyze_information(info: str) -> str:
    """Analyze and summarize information."""
    return f"Analysis: {info} contains valuable insights and detailed information about the topic."

def research_node(state: ResearchState):
    """Node that performs research using tools."""
    print(f"🔍 Research iteration {state['current_iteration']}: {state['research_topic']}")
    
    # Search for information
    search_query = f"research about {state['research_topic']}"
    search_results = search_web(search_query)
    
    # Analyze the results
    analysis = analyze_information(search_results)
    
    # Add to research notes
    state["research_notes"].append({
        "iteration": state["current_iteration"],
        "query": search_query,
        "results": search_results,
        "analysis": analysis
    })
    
    # Add research message
    state["messages"].append(AIMessage(content=f"Research iteration {state['current_iteration']} completed: {analysis}"))
    
    return state

def evaluate_node(state: ResearchState):
    """Node that evaluates the quality of research gathered."""
    print(f"📊 Evaluating research quality...")
    
    # Simple evaluation based on research notes
    total_notes = len(state["research_notes"])
    quality_score = min(0.9, 0.3 + (total_notes * 0.1))
    
    state["evaluation_score"] = quality_score
    
    # Add evaluation message
    evaluation_msg = f"Research evaluation: {quality_score:.2f} quality score based on {total_notes} research iterations"
    state["messages"].append(AIMessage(content=evaluation_msg))
    
    print(f"📈 Quality score: {quality_score:.2f}")
    
    return state

def should_continue(state: ResearchState) -> Literal["research", "synthesize", "end"]:
    """Decide whether to continue researching, synthesize, or end."""
    print(f"🤔 Deciding next action...")
    
    # Check if we've reached max iterations
    if state["current_iteration"] >= state["max_iterations"]:
        print("⏰ Max iterations reached, synthesizing...")
        return "synthesize"
    
    # Check if quality is sufficient
    if state["evaluation_score"] >= 0.8:
        print("✅ Quality sufficient, synthesizing...")
        return "synthesize"
    
    # Continue researching
    print("🔄 Quality insufficient, continuing research...")
    state["current_iteration"] += 1
    return "research"

def synthesize_node(state: ResearchState):
    """Node that synthesizes all research into a final report."""
    print("📝 Synthesizing final research report...")
    
    # Create comprehensive report
    report = f"""
    # Research Report: {state['research_topic']}
    
    ## Summary
    Based on {state['current_iteration']} research iterations, here's what I found:
    
    """
    
    # Add findings from each iteration
    for i, note in enumerate(state["research_notes"], 1):
        report += f"### Iteration {i}\n{note['analysis']}\n\n"
    
    report += f"""
    ## Final Evaluation
    Quality Score: {state['evaluation_score']:.2f}
    Total Iterations: {state['current_iteration']}
    
    ## Conclusion
    This research provides comprehensive coverage of {state['research_topic']} with detailed analysis and insights.
    """
    
    # Add final message
    state["messages"].append(AIMessage(content=report))
    state["research_complete"] = True
    
    print("✅ Research synthesis completed!")
    
    return state

def create_iterative_research_agent():
    """Create an iterative research agent with tracing."""
    # Create LLM (without callbacks - will be passed at graph level)
    llm = ChatOpenAI(
        model="gpt-4",
        temperature=0.7
    )
    
    # Create the graph
    graph = StateGraph(ResearchState)
    
    # Add nodes
    graph.add_node("research", research_node)
    graph.add_node("evaluate", evaluate_node)
    graph.add_node("synthesize", synthesize_node)
    
    # Add edges
    graph.add_edge("research", "evaluate")
    graph.add_conditional_edges(
        "evaluate",
        should_continue,
        {
            "research": "research",
            "synthesize": "synthesize",
            "end": END
        }
    )
    graph.add_edge("synthesize", END)
    
    # Set entry point
    graph.set_entry_point("research")
    
    return graph.compile()

def run_iterative_research():
    """Run the iterative research agent with tracing."""
    print("=== Iterative Research Agent Tracing ===")
    
    # Create callback handler
    callback_handler = NoveumTraceCallbackHandler()
    
    # Create the agent
    agent = create_iterative_research_agent()
    
    # Run with callbacks via config (recommended approach)
    result = agent.invoke(
        {
            "messages": [HumanMessage(content="Research artificial intelligence and its applications")],
            "research_topic": "artificial intelligence and its applications",
            "research_notes": [],
            "evaluation_score": 0.0,
            "max_iterations": 3,
            "current_iteration": 1,
            "research_complete": False
        },
        config={
            "callbacks": [callback_handler],
            "tags": ["iterative_research"]
        }
    )
    
    print(f"\n🎉 Research completed!")
    print(f"📊 Final quality score: {result['evaluation_score']:.2f}")
    print(f"🔄 Total iterations: {result['current_iteration']}")
    print(f"📝 Research notes: {len(result['research_notes'])}")
    
    return result

if __name__ == "__main__":
    run_iterative_research()
    noveum_trace.flush()

📋 Prerequisites

pip install noveum-trace langchain-openai langgraph python-dotenv

Set your environment variables:

export NOVEUM_API_KEY="your-noveum-api-key"
export OPENAI_API_KEY="your-openai-api-key"

🔧 How It Works

1. Iterative Process

The agent follows this flow:

  1. Research: Gather information using tools
  2. Evaluate: Assess the quality of information
  3. Decide: Continue research or synthesize results
  4. Synthesize: Create final report (if quality sufficient)

2. State Management

The ResearchState tracks:

  • Research topic and notes
  • Current iteration count
  • Quality evaluation score
  • Completion status

3. Self-Loop Tracing

Each iteration is traced as a separate span:

  • Research node execution
  • Tool calls and results
  • Evaluation process
  • Decision-making logic

🎨 Advanced Examples

Adaptive Research Agent

def create_adaptive_research_agent():
    """Create an agent that adapts its research strategy."""
    llm = ChatOpenAI()
    
    def adaptive_research_node(state: ResearchState):
        """Adapt research strategy based on previous results."""
        # Analyze previous research to determine next steps
        if state["current_iteration"] > 1:
            # Look for gaps in previous research
            previous_queries = [note["query"] for note in state["research_notes"]]
            # Adapt search strategy based on gaps
            pass
        
        # Continue with research
        return research_node(state)
    
    # Rest of the implementation...

Multi-Source Research

@tool
def search_academic(query: str) -> str:
    """Search academic databases."""
    return f"Academic search results for: {query}"

@tool
def search_news(query: str) -> str:
    """Search news sources."""
    return f"News search results for: {query}"

def multi_source_research_node(state: ResearchState):
    """Research using multiple sources."""
    # Search different sources
    academic_results = search_academic(state["research_topic"])
    news_results = search_news(state["research_topic"])
    web_results = search_web(state["research_topic"])
    
    # Combine results
    combined_analysis = f"""
    Academic: {academic_results}
    News: {news_results}
    Web: {web_results}
    """
    
    # Add to research notes
    state["research_notes"].append({
        "iteration": state["current_iteration"],
        "sources": ["academic", "news", "web"],
        "results": combined_analysis
    })
    
    return state

📊 What You'll See in the Dashboard

After running this example, check your Noveum dashboard:

Trace View

  • Complete iterative workflow
  • Each research iteration as a separate span
  • Tool calls and results
  • Evaluation and decision-making process

Span Details

  • Individual iteration performance
  • Tool execution times
  • Quality score evolution
  • State changes over time

Analytics

  • Iteration patterns and efficiency
  • Quality improvement over time
  • Tool usage statistics
  • Research effectiveness metrics

🔍 Troubleshooting

Common Issues

Infinite loops?

  • Set appropriate max_iterations limit
  • Ensure evaluation criteria are realistic
  • Monitor quality score thresholds

Poor research quality?

  • Adjust evaluation criteria
  • Improve tool implementations
  • Add more diverse research sources

Performance issues?

  • Monitor iteration execution times
  • Optimize tool calls
  • Consider parallel research strategies

🚀 Next Steps

Now that you've mastered iterative research agents, explore these patterns:

💡 Pro Tips

  1. Set iteration limits: Prevent infinite loops with max iteration counts
  2. Monitor quality scores: Track research quality over iterations
  3. Use diverse sources: Combine multiple research tools
  4. Adapt strategies: Modify research approach based on results
  5. Track state evolution: Monitor how state changes through iterations