Most chatbot dashboards open with a big number: conversations this month. It is satisfying to watch, and it tells you almost nothing. It shows the chatbot is switched on. It does not show whether it is helping anyone, finding you customers, or quietly annoying people. At Bolder Digital, we think a small set of numbers does the real work, and the rest is noise.

Start With the Job

Before choosing metrics, decide what the chatbot is for. Most business chatbots do one or more of three jobs:

• capture and qualify leads

• book appointments, tables, or sessions

• answer questions so your team does not have to

The right numbers depend on the job. A chatbot that exists to book appointments should be judged on bookings. One that answers customer questions should be judged on whether the questions get answered. Pick one main outcome and three to five supporting measures. More than that and nobody looks at any of them.

The Numbers That Matter

Think of them in three groups.

Outcomes: did it do the job?

• Goal completion: the share of qualifying conversations that reached the goal, such as a booking made or a lead captured.

• Lead quality: how many captured leads were genuine, and how many became customers.

• Cost or revenue per outcome: what each booking or lead is worth, set against what the chatbot costs.

Quality: did it do the job well?

• Resolution: how often the visitor’s question was actually answered.

• Fallback rate: how often the chatbot gave a “sorry, I can’t help with that” type answer. Vendor guidance often treats under about 15 percent as healthy, but your own trend matters more.

• Handoff rate: how often the conversation went to a person. A high rate is not automatically bad if the handoffs were right, but it signals a training gap when the chatbot is escalating questions it should handle.

• Satisfaction: a simple thumbs up or down on a sample of conversations.

• Repeat contact: the same person coming back within a day with the same issue is a sign the first answer did not work.

• Time to first response: how quickly the bot, and any human behind it, replied.

Insight: what should we fix?

• the most common questions and topics

• the questions it could not answer, which is your list of knowledge gaps

• where conversations drop off

• which pages trigger chats, and what time of day they happen

• how much comes after hours

Why Containment Can Mislead

Containment is the share of conversations handled without a person. Dashboards love it, because a high number looks like savings. The trouble is that a conversation can end without being resolved. The visitor may have given up. Analyses of AI customer service metrics make the point that rising resolution with steady satisfaction is a good sign, while rising containment with falling satisfaction means people are getting stuck in automation, not helped.

So never read containment alone. Pair it with satisfaction, repeat contact, and a look at real conversations.

Vanity Numbers to Ignore

• Total conversations. More chats do not mean more success.

• Messages per chat. Long is not good. It can mean confusion.

• Widget opens. Someone clicked the icon. They may not have said a word.

• Average duration on its own. A short chat that books an appointment is better than a long one that goes nowhere.

A more useful view is a funnel: the widget opened, a message was sent, a lead was captured, the lead was qualified, the customer paid. Each step shows where people are lost.

Set Up the Tracking Properly

A chatbot that captures leads is only measurable if the data goes somewhere useful.

• Send chatbot events to Google Analytics. Common ones are chat opened, chat started, lead captured, and booking made. You can mark the important ones as key events.

• Pass the source into your CRM. Each lead should carry where it came from, such as the page, the campaign, and any tracking tags, and later whether it was won or lost and for how much.

• Check the attribution. Some chat tools do not pass session information into analytics unless they are set up to, which leaves the chats looking like they came from nowhere.

• Ask how they heard about you. Chat leads still count in the “how did you hear about us” answers, so include the question.

Our piece on measuring marketing ROI covers following enquiries all the way to revenue.

Read the Transcripts

The numbers tell you where to look. The transcripts tell you why. A weekly session of about twenty minutes is enough:

• Read ten to twenty real conversations, including some that ended well.

• Mark each problem: a wrong answer, a dead end, a missed lead, a confused visitor, or a handoff nobody picked up.

• Note the questions it could not answer and add them to the knowledge base.

• Fix the biggest problem first, and test the change.

It sounds basic, and it is the single most effective habit we know. Reading what customers actually said is how you find the gaps no dashboard will show.

Segment the Data

Averages hide a lot. Split the numbers by:

• page: a pricing page chat behaves very differently from a blog post chat

• time of day: after hours conversations often have the highest value

• device: mobile visitors type differently and give up sooner

• traffic source: visitors from search versus ads versus social

A chatbot that performs well in one place and poorly in another is easy to fix once you can see it.

Track What Happens After the Handoff

When the chatbot passes someone to a person, the measuring is not finished. How long did it take for a human to reply? How many handoffs were never answered? Slow follow up wastes a good lead, which is why speed matters so much in the research on lead response.

Be Careful With the Data Itself

Transcripts contain personal information. Limit who can see them, set a retention period, and avoid pasting conversations into public AI tools to analyse them. Our piece on chatbot security covers this in more detail.

A Simple One Page Dashboard

For most small businesses, this is plenty:

• conversations started, and the funnel to leads and bookings

• goal completion rate

• fallback rate and the top unanswered questions

• handoff rate and response time to handoffs

• satisfaction on a sample

• leads and customers by source

• cost per customer, set against what the chatbot costs

Review it weekly, with a deeper look monthly.

How Bolder Digital Approaches This

At Bolder Digital, we agree the chatbot’s main job and the numbers that measure it before launch. We set up event tracking and CRM source fields, build a simple dashboard, and read real conversations every week to find gaps. You can see how this fits into our AI Chatbot service.

The Bottom Line

Good chatbot analytics is not a big dashboard. It is a few outcome numbers tied to the chatbot’s job, a few quality numbers that explain them, and a habit of reading real conversations. Ignore the vanity figures, be careful with containment, and use every unanswered question to make the chatbot better.

Frequently Asked Questions

What is the most important chatbot metric?
It depends on the job. For a booking or lead capture chatbot, goal completion, such as bookings made or leads captured, matters most. For a question answering chatbot, resolution and satisfaction matter most.

Is a high containment rate good?
Not on its own. It can mean visitors gave up instead of being helped. Pair it with satisfaction, repeat contact, and a review of real conversations.

What is fallback rate?
The share of messages where the chatbot had no useful answer. A rising rate points to gaps in the knowledge base. Vendor guides often suggest keeping it low, but your own trend is the better guide.

How do I track chatbot leads in Google Analytics?
Send events such as chat started, lead captured, and booking made from the chatbot into Google Analytics, mark the important ones as key events, and pass the campaign and page details through to your CRM.

How often should I review chatbot performance?
A short weekly review of numbers and about ten to twenty transcripts, a deeper monthly look at trends and segments, and a quarterly check that the chatbot’s goals still match your business.

Jarryd Holmes, Founder & Owner, Bolder Digital | Prospect Vale, Tasmania | bolderdigital.com.au