Set Up NovaSynth
Configure the agent profile, managed evaluation resources, and a supported connection.
Before testing, the project needs an agent profile, sufficient NovaSynth credits, at least one persona and scenario, and an active supported connection to a reachable agent.
Creating resources requires synthetic write access. Starting runs requires synthetic execute access.
Complete the first-time wizard
The wizard has two steps.
Agent profile
| Field | Purpose |
|---|---|
| Agent name | Identifies the configured agent in NovaSynth |
| System prompt | Describes the agent behavior and seeds synthetic generation and evaluation context |
| Additional instructions | Adds domain or operating constraints for generated test behavior |
| Agent type | voice, chatbot, or phone |
| Language | Primary language for generated behavior |
The current language selector includes English, Spanish, French, German, Portuguese, Italian, Dutch, Japanese, Korean, Chinese, Hindi, Arabic, Russian, Turkish, Polish, Swedish, Tagalog, and Filipino.
Connection choice
Choose LiveKit, Phone, or HTTP Chat, or skip connection creation and add one later. Saving the profile completes Agent Config, then the selected connection type opens in Test Connections.
Resources created automatically
The first successful Agent Config setup creates or reuses:
- a project dataset for synthetic interactions
- an enabled ETL job with the appropriate managed mapper
- an enabled Eval Job with voice or chatbot defaults
These resources connect successful synthetic trace processing to dataset insertion and scoring. Do not create a second manual pipeline unless the project needs a separate item boundary or scorer configuration.
Add a supported connection
Connections are managed on Test Connections. The tab name describes where connections are configured; there is no universal probe action for every connection type. Verify a connection with a small real run.
LiveKit
| Field | Purpose |
|---|---|
| Name | Identifies the connection in run dialogs |
| LiveKit URL | WebSocket URL for the LiveKit deployment |
| API key and secret | Authorize room creation and agent dispatch |
| Agent name | Worker name NovaSynth should dispatch |
LiveKit supports voice and text runs.
Phone
| Field | Purpose |
|---|---|
| Name | Identifies the destination |
| Phone number | Destination in E.164 format |
| Country code | Calling prefix that must match the number |
Noveum selects its managed caller and SIP infrastructure. Do not enter a SIP trunk. Phone connections are for voice runs.
HTTP Chat
| Field | Purpose |
|---|---|
| URL | HTTP endpoint that accepts a chat request |
| Authentication | None, bearer, API key, basic, or custom header |
| Model | Optional model value used by the endpoint request |
| Agent speaks first | Whether the synthetic conversation starts with the agent |
| Timeout and retries | Request limits for the connection |
| Extra headers | Non-secret headers required by the endpoint |
| Body template | Optional custom request body |
| Response transform | Optional JSON path or transform for extracting the reply |
HTTP Chat supports text runs only. Its API connection test is the only endpoint-level test currently implemented, but a complete NovaSynth run remains the final verification.
Unsupported direct endpoints
Direct Pipecat, WebSocket, VAPI, Retell, and ElevenLabs Conversational execution is not currently available. Their presence in older payload types does not make them worker-supported.
To test a Pipecat-backed agent, expose it through a supported phone number or compatible HTTP Chat endpoint. Production Pipecat tracing is independent of NovaSynth connectivity.
Verify the setup
- create one persona and one scenario
- start one run with a compatible mode and connection
- confirm execution reaches a terminal state
- confirm a synthetic trace ID is present
- wait for analysis to complete
- verify the linked dataset item and scorer results
If execution completes but analysis fails, inspect trace creation, managed ETL, dataset insertion, Eval Job configuration, and scorer errors separately.
Update Agent Config
After initial setup, Agent Config can update the system prompt, additional instructions, conversation sync, and mock-tool settings. Manage connections separately in Test Connections.
Multiple connections can coexist, but one batch uses one endpoint. Run separate batches to compare deployments or providers.
Continue with personas and scenarios or runs and results.