Every voice agent reaches the edge of what it knows. A caller asks something nobody wrote an answer for, or the line is bad, or the booking system times out. What the agent says in that moment decides whether the caller trusts it. At Bolder Digital, we spend as much time writing the “I don’t know” lines as the “yes, we can do that” lines, because that is where a good agent separates itself from a risky one.

Why Guessing Is the Real Danger

A person who does not know something usually hesitates, and you can hear it. An AI can sound just as fluent when it is wrong as when it is right. Voice makes that worse, because once the agent has said something, it cannot be taken back.

Testing guides for voice agents highlight one failure above all others. It is the confident fake confirmation, such as “you’re all booked” after the booking system quietly failed. A related rule appears across the industry. The agent should never confirm a price, a booking, or a policy from memory. It should confirm only from a live lookup, and when it is unsure, it should say so and get a person involved.

So the aim is not an agent that never says “I don’t know.” It is one that says it well.

The Design Rule: Never Leave a Dead End

A good “I don’t know” is never just a refusal. It always comes with a next step. Treat it as a success when it leads somewhere:

• a person who can answer

• a callback

• a message sent by text

• a different way to get what the caller needs

The lines below follow that pattern.

Situation One: It Did Not Hear or Understand

This is the most common one, and the one most often handled badly. Avoid vague lines such as “I didn’t understand.” Conversation design guidance recommends being specific and using what the caller has already said.

• First try: “Sorry, I missed the street name. Could you say that again for me?”

• Second try: “Let me try it another way. Could you spell the street name for me?”

• Third try: “I’m having trouble catching that over the line. I can text you a link to finish this online, or put you through to someone. Which would you like?”

Design guides from Google and Microsoft suggest limiting repeated clarification and handing over by about the third failed attempt. After that, the caller is better served by a person than by the same question again.

Situation Two: The Information Is Not There

The caller asks something outside the approved knowledge.

• “I don’t have that information in front of me, and I’d rather not guess. I can pass the question to the team and have someone call you back, or I can text you a link to our page. Which would suit you?”

This works because it is honest, specific, and gives a choice.

Situation Three: It Is Outside What the Agent Should Answer

Some questions are not a gap in knowledge. They are things the agent should not answer at all, such as advice, anything clinical or legal, or a price that depends on seeing the job.

• “That’s one for our team, and I’m not able to advise on it. I can book you a time with them, or take a message. What would you prefer?”

Do not let the agent soften this into a vague answer, such as “it should be fine”. Say plainly that a person needs to handle it.

Situation Four: The Agent Is Not Sure the Information Is Current

Hours on a public holiday, a changed price, a special event. If there is any doubt:

• “I’m not certain about our hours on that day, and I don’t want to tell you wrong. I can have someone confirm and text you today.”

An honest “not certain” beats a confident wrong answer, especially on days when information changes.

Situation Five: Something Broke on Our Side

If the booking system or calendar is not responding, the agent must not pretend.

• “I’m having trouble with the booking system right now, so I can’t confirm that for you. I’ll take your details and the team will confirm by text as soon as it’s sorted.”

Then it must take the details properly and read them back. The one thing it must never say is that the booking is made when it is not.

Situation Six: The Caller Wants a Person or Asks if It Is an AI

Do not dodge this.

• “I’m an AI assistant. I’m happy to help here, or I can put you through to a person if you’d rather.”

If they ask for a person, give them one, or a firm callback. Fighting it makes things worse.

Situation Seven: Sensitive or Urgent Calls

For emergencies, distressed callers, and anything where wrong wording could cause harm, do not improvise. Use fixed, approved wording written with your team, and hand over straight away. Our guides on healthcare, veterinary, and professional services cover this in more detail.

The Ladder

It helps to think of the agent as climbing a short ladder, in this order:

1. Ask once, specifically.

2. Try a different route, such as rephrasing, spelling, or a text link.

3. Offer a person or a callback.

4. Capture the details and the question, and log it.

If it keeps asking the same thing in the same way, the caller feels trapped. A voice agent that loops is one of the most common ways callers get frustrated.

Make Callbacks Honest

Callback promises are where good intentions turn into broken ones. Only promise what your team will do.

• Do not say “I’ll find out and get back to you” unless there is a process behind it.

• Give an honest time, such as “by the end of the next business day”, only if that is true.

• Collect the name, the number, the best time to call, and the reason, then read the number back.

• Say what happens next, in one sentence.

After hours, adjust it: “The team opens at eight tomorrow, and someone will call you after that.”

Tone: Calm, Brief, and Not Over Apologetic

A few habits keep the tone right:

• Keep it short. A long apology wastes the caller’s time.

• Do not blame them. Avoid “you didn’t say” or “you need to”.

• Skip system language such as “I can’t process that request”.

• Avoid forced humour or chirpiness when something has gone wrong.

• Use plain Australian English, in the words the caller used.

What Never to Say

• “It’s probably fine.”

• “I think it’s around that price.”

• “Don’t worry about that.”

• “I’ll sort that out” when nobody will.

• “You’re all booked” before it is confirmed.

Turn Every “I Don’t Know” Into Improvement

Each unanswered question is a free lesson. Log them, group them, and update the knowledge in a weekly batch, as QA guides recommend. Then test the fix using the caller’s exact words, not your own paraphrase. A rising number of “I don’t know” answers on one topic tells you what to add. A falling number does not mean you are done, because the opposite danger is an agent that has started guessing, so keep reading real transcripts.

Test the Refusals

Part of testing is checking that the agent declines the right things. Build a short list of questions it must not answer, and mark a graceful “I don’t know, here is what happens next” as a pass and a confident answer as a fail. Test it with different accents and background noise, since mishearing is a common source of wrong answers.

How Bolder Digital Approaches This

At Bolder Digital, we write the fallback wording with you, based on your real process for callbacks, handoffs, and after hours. We test every “I don’t know” path, read the unanswered questions each week, and update the agent’s knowledge. You can see how this fits into our AI Receptionist service.

The Bottom Line

A voice agent earns trust by being honest at the edge of what it knows. Ask once and specifically, try another route, offer a person or a callback you can keep, and never confirm anything it has not checked. Then use every gap to make it better. A caller will forgive “I’m not sure, here is what happens next” far sooner than a confident answer that turns out to be wrong.

Frequently Asked Questions

Why is it better for a voice agent to say “I don’t know” than to guess?
Because a confident wrong answer, such as a wrong price or a fake booking confirmation, can cost you a customer and cannot be taken back once spoken. An honest “I’m not sure” paired with a next step protects trust.

How many times should an agent ask the caller to repeat themselves?
Conversation design guidance suggests keeping clarification short, with a different approach on the second try and a handover offer by about the third failed attempt, instead of looping.

What should an agent say when the booking system is down?
It should say plainly that it cannot confirm the booking, take the caller’s details, read them back, and explain that the team will confirm. It must never claim a booking is made when it is not.

Should an agent promise to call people back?
Only if your team will really do it, and only with a timeframe you can meet. Collect the name, number, best time, and reason, and say what happens next.

How should we use the questions the agent could not answer?
Log them, group similar ones, update the knowledge base in a regular weekly batch, and test the fix using the caller’s exact words.

Jarryd Holmes, Founder & Owner, Bolder Digital | Prospect Vale, Tasmania | bolderdigital.com.au