NovaEval

NovaEval measures accuracy, RAG quality, conversation behavior, agent behavior, safety, voice signals, latency, and custom criteria.

Platform Eval Jobs

The platform evaluates output already stored in a dataset:

Dataset items -> Selected scorers -> Item results and aggregates

Production traces require an explicit ETL mapper before they become items. Scorer recommendations propose a starting set; users review and select the final scorers.

Eval Jobs support manual and realtime execution, item filters, sampling, thresholds, stored pass verdicts, errors, and optional reasoning.

Python framework

The novaeval Python package can generate model output inside an experiment:

Dataset -> Target model or models -> Scorers -> Local results

Use it for model comparisons, local experiments, CI, and custom infrastructure.

Start here

• NovaEval overview
• Choose scorers
• Run Eval Jobs
• Prepare datasets