ContextFaithfulnessScorer
Enhanced faithfulness detection with fine-grained claim-by-claim analysis.
Overview
Enhanced faithfulness detection with fine-grained claim-by-claim analysis.
Use Cases
- RAG-based question answering systems
How It Works
This scorer uses LLM-as-Judge technology to evaluate responses. It prompts a large language model with specific evaluation criteria and the content to assess, then analyzes the LLM's judgment to produce a score and detailed reasoning.
Input Schema
| Parameter | Type | Required | Description |
|---|
Output Schema
| Field | Type | Description |
|---|---|---|
| score | any | - |
| passed | any | - |
| reasoning | any | - |
| metadata | any | - |
Score Interpretation
Related Scorers
ConflictResolutionScorer
Evaluates how well an answer handles contradictory information across multiple context chunks. Perfo...
Multi-ContextContextConsistencyScorer
Evaluates consistency of an answer across multiple context chunks. Identifies cases where the answer...
Multi-ContextContextFaithfulnessScorerPP
Enhanced faithfulness detection with fine-grained claim-by-claim analysis. Performs two-stage evalua...
Multi-ContextContextGroundednessScorer
Evaluates how well an answer is grounded in and supported by the provided context. Ensures claims ca...
Frequently Asked Questions
When should I use this scorer?
Use ContextFaithfulnessScorer when you need to evaluate multi-context and rag aspects of your AI outputs. It's particularly useful for rag-based question answering systems.
Why doesn't this scorer need expected output?
This scorer evaluates quality aspects that don't require comparison against a reference answer. It uses the system prompt and context as the implicit ground truth.
Can I customize the threshold?
Yes, the default threshold of 7 can be customized when configuring the scorer.
Quick Info
Ready to try ContextFaithfulnessScorer?
Start evaluating your AI agents with Noveum.ai's comprehensive scorer library.
Explore More Scorers
Discover 112 calibrated LLM-as-Judge scorers for comprehensive AI evaluation