LevenshteinSimilarityScorer
Computes similarity based on Levenshtein (edit) distance between prediction and ground truth. Measures the minimum number of single-character edits needed to transform one string into another.
Overview
Computes similarity based on Levenshtein (edit) distance between prediction and ground truth. Measures the minimum number of single-character edits needed to transform one string into another.
Use Cases
- Accuracy benchmarking and validation
How It Works
This scorer uses deterministic rule-based evaluation to validate outputs against specific criteria. It applies predefined rules and patterns to assess the response, providing consistent and reproducible results without requiring LLM inference.
Input Schema
| Parameter | Type | Required | Description |
|---|---|---|---|
| prediction | str | Yes | Text to compare |
| ground_truth | str | Yes | Reference text |
Output Schema
| Field | Type | Description |
|---|---|---|
| score | float | Similarity score (0-10) |
| passed | bool | True if above threshold |
| reasoning | str | Edit distance analysis |
| metadata.edit_distance | int | Raw edit distance |
| metadata.similarity_ratio | float | Normalized similarity |
Score Interpretation
Default threshold: 7/10
Related Scorers
ExactMatchScorer
Evaluates whether the prediction exactly matches the ground truth. Strictest form of accuracy measur...
NLP MetricsBLEUScorer
Computes BLEU (Bilingual Evaluation Understudy) score between prediction and ground truth. Measures ...
NLP MetricsROUGEScorer
Computes ROUGE scores for text summarization evaluation. Calculates ROUGE-1 (unigram), ROUGE-2 (bigr...
Basic RAGSemanticSimilarityScorer
Computes semantic similarity between query and retrieved context using sentence embeddings. Uses cos...
Frequently Asked Questions
When should I use this scorer?
Use LevenshteinSimilarityScorer when you need to evaluate nlp-metrics and accuracy aspects of your AI outputs. It's particularly useful for accuracy benchmarking and validation.
Why does this scorer need expected output?
This scorer compares the generated output against a known expected result to calculate accuracy metrics.
Can I customize the threshold?
Yes, the default threshold of 7 can be customized when configuring the scorer.
Quick Info
Ready to try LevenshteinSimilarityScorer?
Start evaluating your AI agents with Noveum.ai's comprehensive scorer library.
Explore More Scorers
Discover 106 calibrated LLM-as-Judge scorers for comprehensive AI evaluation