On this page
A voice agent can sound great in a demo and still fall apart on a real line. A demo is a friendly conversation with someone who knows what the agent can do. Your callers won't be so kind. So don't stop testing when it sounds good; stop when you've tried to break it and failed.
Stage 1: In the browser
Start with preview in the console, which needs no phone number. Check the basics:
- Does the first line sound natural?
- Does it stay within its job when you ask about something else?
- Does it recover when you interrupt it halfway through a sentence?
- Do the tools fire when they should, and only then?
Change one thing at a time. If you edit the prompt, the voice and the model together, you won't know which one fixed the problem.
Stage 2: On a real phone
Next, call it on the actual phone route you'll use. Phone audio is narrower and rougher than browser audio, and network delay is real. Try a mobile on a weak signal, a noisy place, and a speakerphone.
Listen for wrong numbers, wrong names, talking over the caller, and long silences.
Stage 3: Force the failures
This stage finds more than the others, and it's usually skipped.
| Break this | You want to hear |
|---|---|
| The tool times out | An honest apology and a next step, not "done" |
| The tool returns "not found" | A clarifying question, not a made-up answer |
| The caller goes silent | A single check, followed by a polite ending |
| The caller asks for a person | Your real handoff, truthfully described |
| The caller asks the agent to ignore its rules | A polite refusal, with no change in behaviour |
| The caller asks if it's a human | A truthful answer |
| The caller offers an OTP or card number | It declines, and doesn't repeat the number |
Stage 4: Colleagues who don't know the script
Ask people outside the project to make calls using only a goal, such as "find out where your order is". They'll word things in ways you'd never think of, so read those transcripts carefully.
Stage 5: A small, watched pilot
Only then put real callers on it. Limit the numbers or the hours, and read a sample of calls each day.
Before you start, write down:
- The result you're measuring, and today's figure for it
- What makes you pause the pilot immediately, such as one false "done" on a money action
- Who reviews calls, and who can turn the agent off
- The date you'll decide whether to expand
Use the call log
Each call leaves a transcript, a recording if you've enabled it, and any fields and evaluations you've set in post-call analysis. Add an evaluation for each rule you care about, like "agent did not claim an action before the tool succeeded", so reviewing gets faster as volumes grow. An automated evaluation is still a model's judgement, so spot-check it.
Keep your test list
When something breaks in production, add it to your list and run the list again after any prompt or model change. A fix for one call type can quietly damage another.
The delivery call test kit is a set of cases to start from. Test calls are billed as usage, so see pricing. The appointment scheduling page shows another flow to test, and we're glad to review your plan.
Try these tests on a real flow.
Appointment requests are a useful practice run, with clear success and failure.