Build a test matrix
Combine reusable personas and scenarios into the exact test cases a NovaSynth run should execute.
A NovaSynth test case pairs one persona with one scenario:
- the persona controls who the synthetic user is and how they communicate;
- the scenario controls the conversation situation and the steps they should attempt.
Build and review these objects separately, then combine only the pairs you want in Runs → New Test Batch.
Recommended sequence
Each persona appears once in the test-case list with the scenarios chosen for it, so a batch never repeats a pair. A batch supports up to 1,000 runs. It does not require a full Cartesian product; include only the combinations that provide useful coverage.
Review the inputs before pairing
- Open Personas and confirm that each selected synthetic user has a clear goal and only the behavior, language, and voice conditions relevant to the test.
- Open Scenarios and review the description, ordered condition and action steps, planned interruptions, and category for each situation.
- In Runs → New Test Batch, choose one persona, select one or more scenarios, then click Add combination.
- Read the Test cases list as the execution plan. Edit or remove a persona's combination before moving on to scorers and the connection.
The run guide describes every wizard step, including scorer selection and critical scorers.
Review run options
| Option | What to verify |
|---|---|
| Batch name | A name that identifies the batch in Runs, reports, and comparisons |
| Test cases | Every listed persona-scenario combination is intentional |
| Scorers | The scorers that should evaluate each run, with the ones that decide pass or fail marked critical |
| Connection | The active staging or test endpoint you intend to exercise; the wizard lists only connections that match the agent type |
| Concurrency | Simultaneous runs from 1 to 20; use 1 for the first connectivity check |
| Max call duration (s) | Per-call voice cap from 60 to 600 seconds; voice mode only |
Next stays unavailable until the current step is valid: a batch name, at least one combination, at least one scorer, and finally a connection with valid concurrency and duration. Review the total run count on Test cases before launching the batch.
Keep coverage interpretable
- Change one dimension at a time when debugging a failure.
- Use tags and descriptive names so the run matrix remains readable.
- Keep a stable regression set and add short-lived exploratory cases separately.
- Start voice batches at low concurrency to expose connection and quota issues safely.