TalkRatioScorer
Agent-to-user speech time ratio from VAD intervals. Metadata includes talk_ratio, user_speech_s, and agent_speech_s.
Overview
Agent-to-user speech time ratio from VAD intervals. Metadata includes talk_ratio, user_speech_s, and agent_speech_s.
Use Cases
- General AI evaluation and quality assessment
How It Works
This scorer uses deterministic rule-based evaluation to validate outputs against specific criteria. It applies predefined rules and patterns to assess the response, providing consistent and reproducible results without requiring LLM inference.
Input Schema
| Parameter | Type | Required | Description |
|---|---|---|---|
| raw_complete_audio | any | Yes | - |
Output Schema
| Field | Type | Description |
|---|---|---|
| score | any | - |
| passed | any | - |
| reasoning | any | - |
| metadata | any | - |
Score Interpretation
Related Scorers
AgentWpmScorer
Agent speaking rate (words per minute) from assistant turn text and VAD-gated agent speech time. con...
Voice & AudioAiInterruptUserScorer
Counts agent barge-in events (agent starts speaking while user utterance is open). Zero events → 10....
Voice & AudioAssistantOverlapScorer
Dual-talk overlap duration from Silero VAD; lower overlap scores higher. Metadata includes overlap_s...
Voice & AudioUserInterruptAgentScorer
Counts user barge-in events (user starts speaking while agent utterance is open). Metadata includes ...
Frequently Asked Questions
When should I use this scorer?
Use TalkRatioScorer when you need to evaluate livekit_conversation and audio aspects of your AI outputs. It's particularly useful for general ai evaluation and quality assessment.
Why doesn't this scorer need expected output?
This scorer evaluates without a reference answer by using the specific input fields listed in its schema. Those fields are evidence for the check; they are not automatically factual ground truth.
Can I customize the threshold?
This scorer's displayed default threshold is 7. Where the eval-job configuration exposes a threshold, review it against the scorer's native scale before saving. The stored pass decision uses that scorer-specific configuration.
Quick Info
Ready to try TalkRatioScorer?
Start evaluating your AI agents with Noveum.ai's comprehensive scorer library.
Explore More Scorers
Discover 112 published scorers across 15 categories: 82 Premium, 30 Free, 76 model-backed, and 36 dedicated voice and audio scorers