Real phone calls
NovaSynth dials your agent over SIP via Twilio, Plivo, or Vobiz — and LiveKit real-time audio — real telephony, real audio, real latency, not a mocked transcript.
SY
Simulation & synthetic data
Break your agents before your customers do.
NovaSynth sends synthetic users at your agents — real SIP phone calls and chat sessions with personas, accents, interruptions, and adversarial intent. It reconstructs whole scenarios and runs your agent (or a candidate fix) to completion with tool calls virtualized — replaying reads, sandboxing writes — so failures surface in rehearsal and fixes are validated on the end-to-end outcome, not just one answer.
Real SIP
calls to any phone number
Voice + chat
one simulation engine
100s
of scenarios per run
app.noveum.ai — call analysis
01 — Capabilities
Synthetic users with real-world chaos.
Personas & accents
Frustrated repeat callers, non-native speakers, rapid talkers — synthetic users with goals, moods, and accents.
Interruptions & barge-in
Talk over the agent mid-sentence, go silent, change topic — the chaos real users bring, on demand.
Adversarial scenarios
Prompt injection, policy probing, out-of-scope requests, and regulatory edge cases — generated systematically.
Chat simulation
The same persona engine drives multi-turn chat sessions over HTTP for text agents.
Regression suites
Save scenario batches and re-run the full personas × scenarios matrix under a shared batch id before every release — a CI gate for agent behavior.
02 — A run
Hundreds of scenarios before breakfast.
Every call is recorded, traced, and scored with the full NovaEval suite — including the audio itself.
Scenario runRUN #88 · 240 SCENARIOS
Frustrated repeat callerThird call about a delayed refundVOICE · SIPPASS
Non-native speakerAddress change with a heavy accentVOICE · SIPPASS
Rapid interrupterCancels mid-sentence, changes mindVOICE · SIPFLAG
Prompt-injection attemptAsks the agent for its system promptCHATPASS
Run #88 — 240 scenarios, 2 flagged: agent spoke over the user after a tool call. Fixed before release.
03 — Under the hood
How a synthetic run actually runs.
An async queue, real telephony and chat endpoints, generated personas and scenarios, and tool virtualization — every synthetic call captured as a full trace and re-scored end-to-end.
Queue architecture
An async queue built on BullMQ, PostgreSQL, and Redis schedules and parallelizes runs — hundreds of scenarios per run without blocking your environment.
Endpoint types
LiveKit real-time audio, Phone via SIP (Twilio, Plivo, Vobiz), and chat over HTTP — the same engine drives voice and text agents alike.
Persona & scenario generation
Personas with goals, moods, and accents are generated alongside scenarios — frustrated repeat callers, non-native speakers, prompt-injection probes — systematically, not hand-authored.
Batch matrix runs
Run the full matrix of personas × scenarios under a shared batch id — coverage across the combinatorial space, grouped and comparable in one place.
Tool virtualization
Reads are replayed and writes are sandboxed, so a changed decision is exercised end-to-end while writes never hit production systems.
Captured & re-scored
Every synthetic call is captured as a full trace and re-scored with the same 106 scorers as production — including the audio itself — so failures surface in rehearsal.
The loop
Part of one closed loop.
Simulation runs feed the same traces and evals as production traffic — so NovaPilot can fix what NovaSynth finds, before launch.
Next step
Put your production AI under control.
Start free and ship your first trace in 15 minutes — or book 30 minutes and we’ll integrate live on the call: your stack, your data, your first eval report before it ends.
SOC 2 Type II · HIPAA · GDPR · On-prem & BYO ClickHouse available