NovaSynth Overview

What is NovaSynth?

NovaSynth is Noveum's synthetic testing system for AI voice and chat agents. Instead of waiting for real users to expose quality issues, NovaSynth lets you:

• Define realistic personas (synthetic users with specific backgrounds, tones, and goals)
• Write scenarios (test scripts covering happy paths, edge cases, and adversarial inputs)
• Run those personas through your live agent at scale — via voice, text, or phone
• Automatically score every session with voice quality, safety, and telephony scorers
• See analytics trends over time to catch regressions early

Every NovaSynth session is a real interaction with your agent. The synthetic user is powered by an LLM following the persona's instructions, so conversations are natural and unpredictable — just like real users.

The Overview tab gives you a live snapshot: total runs, overall pass rate, 70% threshold rate, persona count, and scenario count. The pass rate trend chart shows daily pass rates over the last 7, 30, or 90 days.

The Overview tab

The Overview tab is the landing page when you open NovaSynth. It shows:

From the Overview you can also:

• Run Test → — launch a new batch run directly
• Add Persona — create a new persona quickly
• Browse Library — access the shared persona and scenario library

The lower section shows:

• Runs by Scenario — pie chart of run distribution across scenarios
• Call Duration by Persona — average and max call duration per persona
• Recent Runs — table of recent batch runs with Name, Status, Runs count, Pass Rate, 70% Threshold, Avg Duration, and Created date

Testing modes

NovaSynth supports three session modes depending on how your agent is deployed:

Voice (LiveKit)

The synthetic user joins a LiveKit room and converses with your agent using real-time audio. STT/TTS data, audio quality metrics, and the full conversation are captured.

Best for: Voice assistants, real-time customer support bots, phone-style deployments that use LiveKit.

Text (LiveKit)

The synthetic user connects via LiveKit in text mode — no audio processing, just message exchange. Faster and cheaper than voice, good for validating conversation logic.

Best for: Chatbots, text-first agents, rapid iteration before enabling voice.

Phone (SIP)

NovaSynth places an outbound SIP call to your agent's phone number using a configured SIP trunk. The full call is recorded and analyzed just like a voice session.

Best for: IVR systems, telephony agents, any deployment where the agent answers a real phone number.

How NovaSynth sessions become evaluations

Every completed session automatically:

1. Captures the full conversation transcript and audio (where applicable)
2. Creates StandardData items in your linked dataset
3. Scores those items using your configured scorers (audio quality, telephony, safety, latency)
4. Surfaces results in the run detail view and analytics dashboard

This means NovaSynth tests feed directly into your evaluation pipeline — no manual data preparation needed.

Persona + Scenario
        │
        ▼
NovaSynth session
(voice / text / phone)
        │
        ▼
  StandardData items
  + audio metrics
        │
        ▼
 Scorers run automatically
(MOS, latency, telephony, safety)
        │
        ▼
  Analytics dashboard
  + pass/fail results

Supported platforms

NovaSynth can connect to agents deployed on:

When to use NovaSynth

Getting started

Next steps

• Setup — configure your agent provider and connection
• Personas, Scenarios & Runs — create test cases and analyze results
• Scorers Reference — understand the audio and telephony scorers