Start Free. Scale
As You Grow
One unified credit system powers everything — eval scoring, NovaSynth voice and text testing, and NovaPilot analysis. Start free with 2,500 credits/month and upgrade when you need more.
Plans for Every Stage
Org-flat pricing — one price per organization, not per seat. All plans include NovaSynth voice and text testing.
Growth
7-day trialEnterprise
| Feature | Free | Pro | Growth | Max | Scale | Enterprise |
|---|---|---|---|---|---|---|
| Core Platform | ||||||
| Unlimited projects | ||||||
| API access | ||||||
| Custom org scorers | ||||||
| SSO / SAML | ||||||
| Data retention | 30 days | 90 days | 180 days | 1 year | 2 years | Custom |
| Monthly span capacity | 1M | 20M | 50M | 100M | 500M | Unlimited |
| Storage included | 2 GB | 20 GB | 50 GB | 100 GB | 500 GB | Unlimited |
| Evaluations | ||||||
| Rule-based scorers (always free) | ||||||
| Premium LLM scorers | ||||||
| Datasets & eval jobs | ||||||
| NovaSynth (Voice & Text) | ||||||
| Voice agent testing | Credit-gated | Credit-gated | Credit-gated | Credit-gated | Credit-gated | Credit-gated |
| Text conversation testing | Credit-gated | Credit-gated | Credit-gated | Credit-gated | Credit-gated | Credit-gated |
| NovaPilot | ||||||
| AI trace analysis | ||||||
| Support | ||||||
| Community support | ||||||
| Priority support | ||||||
| Dedicated CSM | ||||||
All plans are per organization (org-flat billing), not per seat.Questions? Talk to us.
One Credit System. Everything Metered Fairly.
Monthly plan credits reset each billing cycle. One-time add-on packs never expire — buy once, use whenever.
per LLM scorer × dataset item
Rule-based scorers always cost 0 credits
per voice call minute
NovaSynth voice agent testing
per text conversation
NovaSynth text agent testing
per analyzed trace
NovaPilot Fix Recommendation report (all plans)
Top Up Anytime — Add-On Credits Never Expire
Need more credits mid-month? Buy a pack once and use it whenever you need it. Packs stack with your plan.
Boost
One-time purchase · add-on credits never expire
5,000 credits · $0.007/credit
Accelerate
One-time purchase · add-on credits never expire
25,000 credits · $0.006/credit
Scale
One-time purchase · add-on credits never expire
100,000 credits · $0.005/credit
Every Dimension That Matters
Your plan covers traces, credits, and access — billed per organization, not per person.
Span Ingestion
Every LLM call, RAG step, and agent action captured as spans — stored for real-time analysis and evaluation.
Noveum Credits
One unified currency for eval scoring, NovaSynth voice/text testing, and NovaPilot analysis.
NovaSynth Testing
Voice and text agent simulation available on all plans — credit-gated, not plan-gated.
NovaPilot Analysis
AI-powered trace analysis and root cause reporting. Requires Growth plan or higher.
Core Features on Every Plan
Core platform features are available on all plans — some advanced features are tier-gated
Deploy Your Way
Choose the deployment model that fits your security, compliance, and infrastructure requirements.
Noveum.ai Managed
Fully managed cloud deployment. We handle everything so you can focus on building great AI.
- Zero infrastructure management
- Automatic updates & scaling
- 99.9% SLA guarantee
Bring Your Own ClickHouse
Use your existing ClickHouse infrastructure. Full control over your data storage.
- Data stays in your infrastructure
- Custom retention policies
- Existing security controls
On-Premise
Full on-premise deployment for maximum control and compliance requirements.
- Complete data sovereignty
- Air-gapped environments
- Custom integrations
Cloud (K8s/Helm)
Deploy on AWS, GCP, or Azure using Kubernetes and Helm charts.
- AWS, GCP, Azure support
- Kubernetes & Helm deployment
- Multi-region available
All deployment options include dedicated support and implementation assistance.
From Sign-Up to Insights in Minutes
No sales calls, no NDAs. Connect your AI application and start seeing results immediately.
Sign Up Free
Create your account in seconds — no credit card required. Your workspace is ready to receive data immediately.
Connect Your AI
Add our SDK in one line of code. Works with OpenAI, Anthropic, LangChain, CrewAI, and any custom stack.
Get Insights
Run eval scoring, test voice agents with NovaSynth, and use NovaPilot to analyze failures — all from your dashboard.
Trusted by AI Teams
See how teams use Noveum to ship better AI, faster.
"We moved from manual spot-checking to automated eval scoring at scale. The credit model means we only pay for actual LLM runs, not seat licenses nobody uses."
Sarah K.
Head of AI Platform, Series C startup
"The free plan was enough to convince our team before we committed. We started testing NovaSynth on day one — no credit card, no friction."
Marcus T.
ML Engineer, Early-stage startup
"NovaSynth let us stress-test our voice agent against hundreds of scenarios before launch. At 25 credits per minute, it's a fraction of what we'd pay for manual QA."
Priya R.
VP Engineering, Voice AI company
Pricing FAQ
Everything you need to know about Noveum credits, plans, and billing
Ready to Start Monitoring Your AI?
Sign up free in under 2 minutes — no credit card required. Connect your first AI application and start tracing LLM calls instantly.
About Noveum.ai — Your 24/7 AI Eval Engineering Team
Noveum.ai is the only platform that doesn't just tell you what's broken — it tells you how to fix it. As your 24/7 AI Eval Engineering Team, Noveum monitors, evaluates, debugs, and sends fixes as PR to you, at scale. Built by ex-AWS SageMaker, ex-Sambanova, and ex-Playo engineers, Noveum is purpose-built for enterprise AI teams running chatbots, voice bots, and autonomous agents in production.
With a lightweight SDK integration (15 minutes or less), Noveum captures everything your AI Agent does — prompts, decisions, tool calls, token usage, context retrievals, and workflow traces. Processing over 6M traces and 60M spans per day, the platform provides production-grade observability at enterprise scale.
The platform includes 100+ specialized AI scorers across 18 categories, including dedicated Audio/TTS Quality and Voice Pipeline Latency scorers — making Noveum the only platform with built-in audio evaluation capabilities. From hallucination detection to mispronunciation scoring, from RAG quality to speaking-over-user detection, every dimension of AI quality is covered.
NovaPilot, Noveum's autonomous AI agent, analyzes evaluation data, identifies failure patterns, tests 136+ prompt variations across 4 evolutionary generations, and delivers verified fixes — all automatically. NovaPilot's 4 specialized agents (Prompt Analyzer, Tool Analyzer, Flow Analyzer, General Analyzer) provide comprehensive coverage of every failure mode. Teams can also create complex custom evaluations based on their Product Requirements.
Noveum supports on-prem deployment with BYO ClickHouse, offers Enterprise SLAs with SOC 2 Type II, HIPAA, and GDPR compliance. Backed by AWS, GCP, DigitalOcean, OVHCloud, Nvidia Inception Program, ElevenLabs, and Anthropic.
Why AI Reliability Matters for Modern Teams
AI Agents are becoming the backbone of digital operations — answering customer questions, triaging support tickets, qualifying leads, routing tasks, summarizing documents, analyzing data, and automating workflows. But unlike traditional software, AI is inherently probabilistic. Even well-designed agents will:
- hallucinate facts
- misunderstand instructions
- misuse tools
- return irrelevant answers
- generate inconsistent formats
- exceed token budgets
- produce partial responses
- violate compliance rules
- degrade over time without monitoring
Unreliable AI leads to increased support burden, lost revenue, compliance risks, poor customer experience, and rising operational costs. Noveum.ai provides the infrastructure needed to measure, monitor, and maintain AI Agent quality at scale.
Noveum Monitoring — Complete AI Observability
Noveum captures all agent behaviors in real time:
- Full prompt & response history
- Tool call inputs and outputs
- Token usage tracking
- Cost attribution per action
- System and user message flows
- Context retrieval logs
- Branching workflow behavior
- Intermediate reasoning
- Agent chain-of-thought traces
- Latency analysis
- Failure modes and exceptions
The Agent Visualizer produces a pre-order execution graph showing how the AI Agent reasons and acts across steps. This visual monitoring layer enables developers, product managers, and operations teams to understand the internal logic of their AI Agents and diagnose complex failure patterns.
Noveum Evaluation — Scoring AI Performance with 100+ Specialized Scorers
Noveum.ai includes the most comprehensive evaluation library available for enterprise AI Agents. Every scorer has been designed to evaluate a different dimension of AI output quality — from hallucination detection and factual grounding to conversation health, safety compliance, RAG retrieval quality, structured formatting, and system performance.
Each evaluation produces:
- numeric score
- natural-language reasoning
- step-by-step justification
- references to context used
- recommendations for improvement
Full Scorer Library With Explanations (100+ Scorers)
1. Accuracy, Matching & Semantic Quality Scorers
- ExactMatchScorer
Checks if the AI output perfectly matches the expected answer. Ideal for classification or deterministic tasks. - AccuracyScorer
Measures the proportion of correct responses across a dataset — essential for baseline LLM quality benchmarking. - MultiPatternAccuracyScorer
Supports multiple correct output forms. Useful for flexible workflows where several answers are acceptable. - F1Scorer
Balances precision and recall to measure how accurately the AI captures all important elements. - BLEUScorer
An n-gram matching metric for evaluating text similarity. Used broadly in summarization and generation tasks. - ROUGEScorer
Measures recall overlap with reference text, especially useful for long-form content evaluation. - LevenshteinSimilarityScorer
Calculates edit distance between expected and generated text, identifying near-matches efficiently. - SemanticSimilarityScorer
Uses embedding-based comparisons to assess meaning similarity, not just surface text similarity. - InformationDensityScorer
Scores how substantive and information-rich an answer is, identifying fluff or low-density output. - ClarityAndCoherenceScorer
Evaluates readability, logical structure, and flow — vital for customer-facing AI responses. - AnswerCompletenessScorer
Checks whether the AI addressed every part of the prompt completely. - QuestionAnswerAlignmentScorer
Ensures direct alignment between user question and generated answer. - TechnicalAccuracyScorer
Validates correctness for domain-specific answers in finance, engineering, legal, or medical tasks. - TerminologyConsistencyScorer
Ensures specialist terminology is used appropriately and consistently in AI responses.
2. Relevance, Faithfulness & Hallucination Scorers
- AnswerRelevancyScorer
Measures how relevant the AI's response is to user intent. - FaithfulnessScorer
Checks whether the AI stays grounded in provided context without adding unsupported claims. - FactualAccuracyScorer
Identifies factual inaccuracies by comparing output with known truths. - HallucinationDetectionScorer
Detects fabricated or invented information. One of the most important scorers for enterprise safety. - ConflictResolutionScorer
Identifies contradictions in the AI's reasoning. - ContextPrioritizationScorer
Ensures the AI selects the most important context pieces when generating a response. - ContextFaithfulnessScorerPP
Validates that the answer is grounded in provided documents. - ContextGroundednessScorer
Checks if every important claim is supported by context. - ContextCompletenessScorer
Measures whether the AI uses enough context to form a complete answer. - ContextConsistencyScorer
Ensures consistency with reference material across statements.
3. RAG (Retrieval-Augmented Generation) Scorers
- ContextualPrecisionScorer
Evaluates relevance of retrieved context snippets used in the final answer. - ContextualRecallScorer
Measures how much relevant content the AI included from the available context. - RAGASScorer
Industry-standard RAG quality scoring across groundedness, relevance, and answer quality. - ContextualPrecisionScorerPP
Enhanced precision evaluation for more complex retrieval chains. - ContextualRecallScorerPP
Advanced recall scoring that checks deeper semantic relevance. - RetrievalF1Scorer
Harmonic mean of precision and recall for retrieval effectiveness. - RetrievalRankingScorer
Evaluates document ranking quality for RAG pipelines. - RetrievalDiversityScorer
Ensures retrieval results include diverse, non-redundant knowledge sources. - AggregateRAGScorer
Provides a combined score summarizing RAG performance. - RAGAnswerQualityScorer
Evaluates whether the final answer is high-quality and well-grounded.
4. Conversation & Role Behavior Scorers
- ConversationRelevancyScorer
Checks turn-by-turn relevance in multi-turn dialogues. - ConversationCompletenessScorer
Ensures the entire user intent has been addressed in the conversation. - RoleAdherenceScorer
Ensures the agent sticks to its assigned role and persona. - ConversationalMetricsScorer
Composite conversational quality scorer measuring flow, engagement, and relevance.
5. Safety, Bias & Harm Prevention Scorers
- AnswerRefusalScorer
Detects whether AI refuses unsafe or disallowed tasks appropriately. - ContentModerationScorer
Flags potentially harmful, explicit, or policy-violating content. - ContentSafetyViolationScorer
Scores violations of safety, compliance, or governance guidelines. - ToxicityScorer
Evaluates presence of insulting, aggressive, or harmful language. - IsHarmfulAdviceScorer
Detects dangerous recommendations — crucial for sensitive domains. - NoGenderBiasScorer
Ensures fair, unbiased responses related to gender. - NoRacialBiasScorer
Evaluates outputs for racial bias or stereotyping. - NoAgeBiasScorer
Checks for age-related prejudice in AI responses. - CulturalSensitivityScorer
Ensures the AI respects cultural norms and avoids offensive phrasing.
6. Structured Output & Format Scorers
- IsJSONScorer
Validates whether responses follow correct JSON formatting. - IsEmailScorer
Checks for valid email generation structure. - ValidLinksScorer
Ensures URLs in the output are properly formatted and valid.
7. Multi-Judge & Deep Reasoning Scorers
- PanelOfJudgesScorer
Uses multiple LLM 'judges' to provide averaged evaluations for robust scoring. - SpecializedPanelScorer
Domain-specific multi-judge scorer for technical or expert evaluations. - GEvalScorer
A generalized evaluation approach measuring reasoning, structure, and coherence. - AgentScorers
Bundle of agent-specific evaluations optimized for multi-step AI workflows.
8. Pipeline & System-Level Scorers
- QueryProcessingEvaluator
Evaluates how accurately the system interprets and processes user queries. - RetrievalStageEvaluator
Scores the retrieval portion of a RAG pipeline. - RerankingEvaluator
Evaluates how well documents are reranked for relevance. - RAGPipelineEvaluator
Full end-to-end scoring for retrieval + ranking + generation. - PipelineCoordinationScorer
Evaluates coordination of pipeline components across stages.
9. Latency, Resource & Error Scorers
- LatencyAnalysisScorer
Measures latency and identifies bottlenecks across the AI execution path. - ResourceUtilizationScorer
Assesses token efficiency, compute usage, and cost impact. - ErrorPropagationScorer
Detects how errors propagate across multi-step agent workflows.
10. Knowledge & Memory Scorers
- KnowledgeRetentionScorer
Measures whether an AI preserves key information across turns or steps.
Noveum's Auto-Fixing Engine
After evaluating agents using all scorers above, Noveum automatically analyzes failures and runs improvement experiments such as:
- prompt optimization
- model selection for better cost-performance
- parameter tuning
- workflow adjustments
- tool-call corrections
The system then provides a full recommendation report or can automatically generate pull requests to update your agent logic.
AI Cost Optimization With Noveum
Noveum identifies token-heavy prompts, inefficient retrieval patterns, expensive models, and redundant workflow steps. It analyzes cost per generation step and suggests cheaper, higher-performing alternatives.
Companies typically see 20-50% reduction in AI operational cost after implementing Noveum's recommendations.
Noveum.ai — Your 24/7 AI Eval Engineering Team
As companies scale AI Agents to hundreds and thousands, Noveum provides the only platform that monitors, evaluates, and autonomously fixes broken agents at scale. Built for enterprise from day 1, with on-prem deployment, BYO ClickHouse, and SOC 2 Type II compliance. Trusted by agent builders, banks, and telecom providers.