ConversationFlowAdherenceScorer
LLM judge that scores whether the current assistant response follows expected flow from the system prompt and conversation history (0-10 JSON score + reasoning).
Overview
LLM judge that scores whether the current assistant response follows expected flow from the system prompt and conversation history (0-10 JSON score + reasoning).
Use Cases
- Conversational AI quality assessment
How It Works
This scorer uses LLM-as-Judge technology to evaluate responses. It prompts a large language model with specific evaluation criteria and the content to assess, then analyzes the LLM's judgment to produce a score and detailed reasoning.
Input Schema
| Parameter | Type | Required | Description |
|---|---|---|---|
| message | any | Yes | - |
| output_text | any | Yes | - |
| system_prompt | any | Yes | - |
| agent_response | any | Yes | - |
| conversation_context | any | Yes | - |
Output Schema
| Field | Type | Description |
|---|---|---|
| score | any | - |
| passed | any | - |
| reasoning | any | - |
| metadata | any | - |
Score Interpretation
Related Scorers
RepetitionFuzzyMatchScorer
Rule-based repetition scorer using difflib SequenceMatcher similarity between the current assistant ...
ConversationalConversationRelevancyScorer
Assesses how relevant each response is to the current conversation context and user query. Evaluates...
ConversationalConversationalMetricsScorer
A composite scorer that combines multiple conversational evaluation metrics into a single comprehens...
ConversationalConversationalRoleAdherenceScorer
Measures how well an AI maintains its assigned role or persona throughout a conversation. Evaluates ...
Frequently Asked Questions
When should I use this scorer?
Use ConversationFlowAdherenceScorer when you need to evaluate repetition_and_flow and conversational aspects of your AI outputs. It's particularly useful for conversational ai quality assessment.
Why doesn't this scorer need expected output?
This scorer evaluates without a reference answer by using the specific input fields listed in its schema. Those fields are evidence for the check; they are not automatically factual ground truth.
Can I customize the threshold?
This scorer's displayed default threshold is 7. Where the eval-job configuration exposes a threshold, review it against the scorer's native scale before saving. The stored pass decision uses that scorer-specific configuration.
Quick Info
Ready to try ConversationFlowAdherenceScorer?
Start evaluating your AI agents with Noveum.ai's comprehensive scorer library.
Explore More Scorers
Discover 112 published scorers across 15 categories: 82 Premium, 30 Free, 76 model-backed, and 36 dedicated voice and audio scorers