Answer Quality ScorerLLM-as-Judge

AnswerCompletenessScorer

Evaluates completeness and coverage of AI-generated answers. Assesses whether the answer addresses all aspects of the question and uses available context effectively.

Overview

Evaluates completeness and coverage of AI-generated answers. Assesses whether the answer addresses all aspects of the question and uses available context effectively.

qualityragconversationalllm-judgetrace-evaluationcoveragethoroughness

Use Cases

  • RAG-based question answering systems
  • Conversational AI quality assessment

How It Works

This scorer uses LLM-as-Judge technology to evaluate responses. It prompts a large language model with specific evaluation criteria and the content to assess, then analyzes the LLM's judgment to produce a score and detailed reasoning.

Input Schema

ParameterTypeRequiredDescription
output_textstrYesAnswer to evaluate for completeness
input_textstrYesOriginal question
contextdict | str | listNoAvailable context

Output Schema

FieldTypeDescription
scorefloatCompleteness score (0-10)
passedboolTrue if complete
reasoningstrCoverage analysis
metadata.covered_aspectslistAspects addressed
metadata.missing_aspectslistAspects missing

Score Interpretation

Default threshold: 7/10

9-10ExcellentResponse fully meets all evaluation criteria
7-8GoodResponse meets most criteria with minor issues
5-6FairResponse partially meets criteria, needs improvement
3-4PoorResponse has significant issues
0-2FailingResponse fails to meet basic criteria

Frequently Asked Questions

When should I use this scorer?

Use AnswerCompletenessScorer when you need to evaluate quality and rag aspects of your AI outputs. It's particularly useful for rag-based question answering systems.

Why doesn't this scorer need expected output?

This scorer evaluates without a reference answer by using the specific input fields listed in its schema. Those fields are evidence for the check; they are not automatically factual ground truth.

Can I customize the threshold?

This scorer's displayed default threshold is 7. Where the eval-job configuration exposes a threshold, review it against the scorer's native scale before saving. The stored pass decision uses that scorer-specific configuration.

Quick Info

CategoryAnswer Quality
Evaluation TypeLLM-as-Judge
Requires Expected OutputNo
Default Threshold7/10

Ready to try AnswerCompletenessScorer?

Start evaluating your AI agents with Noveum.ai's comprehensive scorer library.

Explore More Scorers

Discover 112 published scorers across 15 categories: 82 Premium, 30 Free, 76 model-backed, and 36 dedicated voice and audio scorers