Platform Overview
Choose the Noveum interface for tracing, evaluation, synthetic testing, and agent improvement.
Noveum connects captured application behavior to versioned evaluation data. Use the platform to answer four questions:
- What did the application do?
- Which behaviors fail the acceptance criteria?
- What change is most likely to improve them?
- Did the candidate change work before and after release?
Platform surfaces
| Surface | Use it for |
|---|---|
| Project Traces | Filter requests by project, environment, version, status, and time; inspect the spans and payload behind a result. |
| Datasets | Store normalized, versioned items used by evaluation and regression workflows. |
| ETL jobs | Convert rich production traces into the schema required by scorers. |
| NovaEval | Apply compatible scorers and inspect per-item reasoning and aggregate results. |
| NovaPilot | Group recurring failures and prioritize changes. |
| NovaSynth | Exercise an agent with repeatable personas and scenarios. |
Send application behavior
Choose the ingestion surface that fits the application:
All three produce the same trace and span model. Preserve model inputs and outputs, system prompts, token usage, tool definitions and results, retrieval queries and context, stable identifiers, status, and timing so ETL can build complete evaluation items.
Start from an existing dataset
Production tracing is optional for the dataset-first workflow. Import or create normalized items, run NovaEval, use NovaPilot to plan a change, and validate it with NovaSynth. Add tracing later when production evidence and release comparisons are required.
See Choose an Evaluation Workflow for both paths.
Agent-assisted workflows
- Noveum Agent Skill guides instrumentation and trace completeness in a code repository.
- MCP server reference documents resources and prompts for traces, datasets, ETL, evaluation, NovaPilot, and NovaSynth.