Test a Voice Caller Who Changes Their Mind
Shashank Agarwal
A caller can change a preference before an operation is confirmed. The voice agent needs to update the pending task without carrying an outdated approval into a different action.
Quick answer: Give the scenario an initial request, an explicit revision and a defined commitment boundary. Check the proposed and applied operation against the latest established intent. Preserve earlier state in the trace, but ensure it does not silently override the caller's correction.
Put the change before commitment
Consider an illustrative appointment call. The caller first asks for Tuesday, then requests Thursday before accepting a proposal.
The expected behavior is to investigate Thursday and communicate the resulting option. If no Thursday slot exists, the agent should explain that limit rather than silently return to Tuesday.
Define the states to inspect
| State | Evidence needed |
|---|---|
| Initial request | Original preference |
| Correction | New preference and when it was supplied |
| Current proposal | What the agent now offers |
| Confirmation | Which proposal the caller accepted |
| Operation | What was actually submitted or applied |
These are synthetic fixture states, not real appointment records.
Test related but different cases
Include an unambiguous correction, an ambiguous statement and a change after the operation has already completed.
The last case may require a separate change or cancellation workflow. It should not be graded as though the first operation never occurred. Define the application's behavior rather than expecting the agent to rewrite history.
Inspect confirmation scope
An earlier “yes” may apply to a previous proposal. The test should establish which action was accepted before evaluating whether a later operation was permitted.
Use the operation arguments and available state evidence. A final answer that mentions Thursday can conceal a tool request that still contains Tuesday.
Keep the regression bounded
After a candidate change, retain a case where the caller never changes their mind. The agent should not continually reopen a settled preference without evidence of a correction.
I'm Shashank Agarwal, founder and CEO of Noveum AI and creator of API.market. NovaSynth supports configured synthetic-user testing; the scenario and acceptance conditions remain part of the team's design.
See the caller-goal guide for task setup and the failed-simulation guide for the evidence to retain.
Follow my work on LinkedIn. Drafted with AI assistance.
Get Early Access to Noveum.ai Platform
Join the select group of AI teams optimizing their models with our data-driven platform. We're onboarding users in limited batches to ensure a premium experience.