How to test an AI agent before it talks to customers
A practical checklist for testing AI agents before launch: realistic scenarios, edge cases, tool permissions, handoff checks and release gates you can repeat.
By the Zoft team · · 2 min read
An AI agent that works in a demo can still fail on a real customer. Real people interrupt, change their minds, give half an address and ask things nobody planned for. Testing is how you find that out before they do.
These tests are often called evals: a set of scenarios the agent must pass, run every time something changes. The word is new, the idea is not. It is the same as testing software, applied to a system that makes choices.
1. Write the outcome and the pass condition
Start with one sentence about what success looks like, for example: a caller asks for an after-hours appointment and ends the call with a booked slot and a confirmation. If you cannot say what a pass looks like, you cannot test it.
2. Build realistic scenarios
Collect real examples from your own calls and messages, and write them as test cases. Cover:
- The main path: the easy, typical request.
- Variations: different wording, accents, names, and ways of giving a date.
- Messy callers: interruptions, changes of mind, background noise, off-topic questions.
- Missing information: no phone number, an unclear address, a vague problem.
- Out-of-scope requests the agent should decline or pass on.
3. Test the tools, not just the talking
Check what the agent actually changed. Did the booking land on the right day? Was the record created once and not twice? Did it use only the tools it was meant to? An agent can sound perfect and still write the wrong thing to your calendar.
4. Test the limits and the handoff
- Ask for a person and confirm the handoff happens, with a useful summary.
- Push past a limit, such as asking for a large refund, and confirm it escalates.
- Try to talk the agent out of its rules and confirm it holds.
- Raise a safety or legal topic and confirm it follows the fixed rule you wrote.
5. Make it a release gate
Run the same set every time you change the instructions, the tools or the model. If a scenario that used to pass now fails, the change does not ship until it is understood. Over time you add every real failure to the set, so the same mistake cannot return quietly.
6. Keep a record after launch
Testing does not end at go-live. Keep a record of each run: what was said, which tools were called and what changed. Review a sample regularly, especially in the first weeks, and turn surprises into new tests. See human handoff for AI agents for the cases to watch.
How Zoft supports this
Zoft's evals and controls are shared across every product: permissions, handoff rules, release gates and run records. You can review and edit each generated part of an agent before it goes live, and check it against your scenarios first. See also what is an AI voice agent for the call-handling case.
Frequently asked questions
- What are AI agent evals?
- Evals are a repeatable set of test scenarios an AI agent must pass, covering the main path, messy variations, tool changes, limits and handoff, run every time the agent changes.
- How do I test an AI agent before launch?
- Define the outcome and pass condition, build scenarios from real calls and messages, check what the agent actually changed in your tools, test limits and handoff, and rerun the whole set on every change as a release gate.
- Should I keep testing after an AI agent goes live?
- Yes. Keep a record of each run, review a sample regularly, especially in the first weeks, and add every real failure to your test set.