Noveum.ai

Trace, evaluate, and act on evidence

The closed-loop evaluation layerfor production AI agents.

Turn sampled production traces into dataset items, apply selected compatible scorers, investigate the evidence, and organize repeated failures into recommendations your team can verify.

Book a demoStart free

Free tier · guided setup to your first trace

NovaSynth by Noveum - Test your voice agent on the callers you can’t stage. | Product Hunt

SOC 2 Type II in progress · GDPR requirements review · enterprise deployment options

agent.py
from noveum_trace.integrations.langchain import (
    NoveumTraceCallbackHandler,
)
 
handler = NoveumTraceCallbackHandler()
 
agent.invoke({"input": question},
    config={"callbacks": [handler]})

Latest Noveum activity

Traced 1,248 agent runs today
NovaEval flagged hallucination_rate · 7.1%
NovaPilot report ready · 4 issues · 2 recommendations

Instrument the agent, configure an evaluation, and organize the resulting evidence.

Open-source SDKPython · REST ingest · OpenTelemetry-aligned
High volumeproduction trace ingestion
SOC 2 Type IIIn progress · GDPR

In production at

DarGlobalMyOperatorand AI platform teams across telecom, real estate & financial services

Works with your stack

OpenAIAnthropicLangChainLangGraphLiveKitPipecatCrewAIOpenTelemetry

One unified platform

One AI agent evaluation platform, from trace to recommended action

Most monitoring tools stop at an alert. Noveum connects the failing signal to traces, scorer evidence, synthetic runs, and ranked recommendations so engineers can decide and verify what to change.

Observability

noveum-trace

The open-source Python SDK records instrumented LLM calls, tool invocations, retrievals, and agent steps as structured spans. Add custom spans and attributes for application-specific work; supported integrations include LangChain, LangGraph, LiveKit, Pipecat, and CrewAI.

  • Deep framework integrations
  • Threads & multi-agent graphs
  • Fail-soft transport

High-volume tracing · OpenTelemetry-aligned · Apache-2.0 SDK

noveum.ai/app - noveum-trace Explorer
Authenticated noveum-trace Explorer with trace search, a nested span flow, and selected retrieval attributes

Customers

What our customers say.

DarGlobalSwitched from Langfuse
“Being able to analyze spans and traces directly inside the platform, and identify issues right there, has saved a lot of debugging time. Before, we manually went through logs, copied traces into Claude Code, analyzed issues, then implemented fixes.”
Maintenance and deployment overhead significantly reduced after switching from Langfuse
AI-assisted debugging directly on spans and traces, inside the platform
AI-suggested scorers based on the logs and traces actually captured
CrewAI integration requested as a custom requirement: shipped and integrated quickly

AI Engineering Team · DarGlobal

MyOperatorVoice agents at scale
“We went from spending days debugging agent failures to reviewing prioritized evidence and recommendations in one place.”

84→95%+

call success rate

200×

faster iteration

4 to 6×

answer quality

10 min

trace → ranked recommendation

Lead AI Engineer, MyOperator

Read the case study

Evaluate any agent

One platform for evaluating chat, voice, and workflow agents

Chat agents

Turn selected production traces into structured dataset items, choose compatible scorers from the published library, and compare configured evaluation results.

Chatbot evaluation

Voice agents

Transcripts miss what broke on a call. Exercise a configured voice agent with reusable personas and scenarios, then apply selected compatible checks from 36 voice and audio scorers.

Voice agent evaluation

Workflow agents

Multi-step agents fail between the steps. Evaluate tool calls, routing, and outcomes across the whole workflow.

Workflow agent evaluation

Enterprise-ready

Your agents, your data, your infrastructure

Scope in-VPC, on-premise, or BYO ClickHouse options with Noveum. Review the resulting data flows, SSO, access-control, audit, and operational boundaries for the selected agreement.

SOC 2 Type II in progressGDPR requirements reviewSSO / SAML optionsorganization access controlsauditability reviewBYO ClickHouse optionon-premise option

Designed for high-volume trace ingestion · on-premises and BYO ClickHouse options

Explore enterprise

FAQ

The questions enterprise teams ask first.

What is Noveum?

Noveum is an AI agent evaluation platform for production chat, voice, and workflow agents. It records instrumented runs, provides a library of 112 published scorers, supports configured voice and chat testing, and turns evaluated failure evidence into ranked recommendations.

What makes NovaPilot different?

NovaPilot analyzes a selected evaluated dataset and produces a structured report: overall health, scorer performance, recurring failure patterns, system-prompt findings, and ranked recommendations with confidence and effort. It recommends changes; your team implements and verifies them.

Can I run Noveum on-prem?

On-premise, in-VPC, and BYO ClickHouse options can be scoped for qualifying enterprise engagements. Architecture, managed dependencies, data flows, available controls, and operational ownership depend on the agreement. SOC 2 Type II is in progress.

Does NovaPilot change my application automatically?

No. NovaPilot produces recommendations backed by evaluated items, traces, and scorer results. Your engineers decide what to change, implement it in your application, and run new evaluations or synthetic tests to verify the outcome.

Next step

Put your agents in the loop

Production AI agents your customers can trust.

Talk to salesStart free

Trusted in production by enterprise and high-growth teams