Guide 01
Most of what is written about AI customer support comes from companies selling it, which makes it hard to learn what actually happens when you add a support chatbot to a website. This guide gives the explanation we would give a customer: which questions a bot can reliably answer, which it should never handle, and how to tell within a month whether it is worth keeping.
Before evaluating any tool, read your last two hundred support conversations and sort them into two piles.
The first pile is lookups, such as "What are your hours?", "Do you take my insurance?", "Where is my order?", "How much is a deep clean?", "Can I book Thursday?" and "Do you serve my zip code?" Each has one correct answer that does not change based on who is asking, and the person answering usually reads it from a page or a screen.
The second pile is judgment: messages such as "My order arrived broken and I am angry", "Can you make an exception?", "Which of these two options is right for my situation?" and "I want to cancel and I want you to talk me out of it." These require someone to weigh the situation, and often to take responsibility for a decision.
The size of each pile sets the upper limit on what automation can do for you. Restaurants, clinics, home services companies and online shops usually find that the first pile holds most of their messages, because the questions that reach a small business website are mostly practical. A consultancy, or a company selling a complex product to other businesses, often finds the opposite. Neither kind of business is doing anything wrong.
Vendors often quote deflection rates (the share of questions a bot answers without a person stepping in) as if the software alone decided them. In practice, the rate depends mostly on your customers' questions. Anyone who quotes a number before seeing your support tickets (the requests your customers send) is guessing, and that includes us.
For the detailed figures, see our articles on how much of a ticket queue a chatbot actually removes and on which categories of ticket automate cleanly and which do not.
These are listed roughly by how reliably they work, not by how impressive they look in a demonstration.
Two of these are covered in their own articles. Automating the questions you answer every day is the fastest to get right, and covering the hours nobody is staffed for usually does the most to improve customers' opinion of you, because otherwise they would get no reply at all.
This list matters more than the one above, because these mistakes cost more than the successes are worth.
Getting the handoff right matters more than anything else on this page. If a bot answers eight questions well and then traps a customer on the ninth, the customer will remember the ninth.
Four situations should always bring in a person, without any argument from the bot:
How the handoff feels matters as much as when it happens. The customer should not have to repeat themselves, so whoever picks up should already have the transcript (the written record of the conversation). If nobody is available, say so plainly, give a real timeframe, and ask how to reach the customer later. "Our team is offline until 9am, leave your number and you will get a call before ten" is a good outcome. Promising a reply and sending nothing is worse than making no promise.
A support bot is only as good as the information it has been given, and keeping that information accurate is most of the ongoing work.
You need fewer answers at the start than most people expect. Your top ten questions, answered the way your best staff member answers them, cover a large share of your messages. Do not begin by writing two hundred answers. Begin with ten, then watch for the questions the bot cannot handle.
After that, the work is mostly reading. Once a week, read the conversations where the bot failed or handed off, and ask one specific question: was this a gap in what the bot knows, or a question it should never have answered? Fill the gaps, and add the second kind of question to your handoff rules. Businesses that do this weekly for a month usually end up with a bot that works well. Those that launch it and stop paying attention keep a bot that is no more useful than it was on the first day.
One caution about tone: an answer that is technically correct but reads like a policy document usually prompts a follow-up question, which means the question was not really answered. Write the answers the way a helpful colleague would say them out loud.
If you use an older support ticket system, this step often depends on it, and we cover that separately in modernizing a help desk without replacing everything at once.
Deflection rate is the number most chatbot reports show first, and it is the easiest one to make look good. A bot that makes it hard to reach a person will show a high deflection rate while it loses you customers. Read these four numbers together instead.
Alongside the numbers, read twenty transcripts a week yourself. The numbers show where a problem is and how large it is. The transcripts show what is actually going wrong, and they often reveal problems no report would flag.
Choosing what the bot knows and keeping it current is a larger subject, covered in the guide to chatbot knowledge and measurement, along with where wrong answers actually come from and which numbers are worth reading afterwards.
This order avoids the most common mistake, which is launching with an enormous knowledge base (the collection of written answers and business information the bot draws on), no clear handoff rules and no plan to read the transcripts. A bot set up that way answers many questions slightly wrong, and customers learn to ignore the widget, meaning the chat window on your site.
These three patterns explain most of the support bots that are quietly switched off six months after launch.
The bot that will not let go. Someone set it up to maximize deflection, so every route to a person is hidden behind "let me try to help with that first". The deflection numbers look excellent. Customers learn that the chat window will not help and start emailing the sales address instead, which quietly becomes a support inbox that nobody measures. The warning sign is fewer chat conversations and more messages arriving through other channels.
The bot nobody checks. It launched, it works about as well as it did on the first day, and nobody has opened a transcript since. It answers the questions it was given and fails without notice on everything else. This pattern does little harm, but the bot becomes steadily less useful. The fix is thirty minutes every week, which is all it takes.
The bot that answers too confidently. When it is asked something outside its knowledge, it gives an answer that sounds right instead of admitting the gap. In support, this is worse than no answer, because the customer acts on it and the correction comes later, when they are angrier. A bot that says "I do not have that, let me get someone who does" is doing its job. Before launch, test for this on purpose by asking questions you know it has no answer for.
The software cost is the easier half to estimate, and usually the smaller one. The main costs are an afternoon of setup, thirty minutes a week of reading, and one person who is responsible for the bot. A bot with no owner usually falls into the second failure pattern within a quarter.
When you estimate the savings, be careful which number you claim. Hours saved is the number most people use first, and it is the least reliable, because the time freed up rarely turns into anything you can measure. Two numbers are more reliable:
If your inbox is mostly judgment calls, neither number will be large, and a support bot will be a modest improvement rather than a major change. That is a reasonable result, and it is better to learn it in week one than after a full launch.
Support is usually the first job a business gives a chatbot, because the problem is obvious and the questions repeat. The same chat window can also do two other jobs, and the lines between the three are less clear than the labels suggest.
A conversation that starts with "do you take my insurance" is a support question, and it is also often a new patient deciding where to book. One that starts with "where is my order" is support, and the reply is a chance to keep a customer who might leave after one more bad experience. Treating the chat window only as a way to deflect questions measures half of what it does and misses the half that affects revenue.
This has two practical consequences. First, do not put the support bot only on a help page. Questions also come up on service, pricing and product pages, because that is where people hesitate before deciding. Second, when a support conversation turns into an intention to buy or book, the bot should continue the conversation rather than answer the question and stop. Collecting that interest is a separate job with its own rules, and it is covered in our guide to chatbot lead capture.
The reverse is also true. A lead capture bot that cannot answer a basic practical question loses the potential customer at the moment it mattered most. In practice, most small businesses want one conversation that can do both jobs. This is simpler than it sounds, because both jobs draw on the same ten answers you wrote in week one.
Two related questions come up often enough to have their own articles. On whether to use a chatbot or live chat answered by your staff, see chatbot or live chat, and when each is the wrong choice. On how faster answers affect whether customers stay with you, see what support response time does to churn.
It depends mostly on the kind of questions you receive, not on how good the bot is. If most of your messages ask about hours, pricing, order status or booking, a bot can answer a large share of them, because those are lookups. If most need a judgment call, a bot can answer very few and should not try. Sort your last two hundred tickets before you estimate anything.
Customers are usually annoyed by a chatbot that will not let them reach a person, which is a different problem. A bot that answers right away is better than a contact form, and one that hands off in one step is better than a phone queue. Most complaints are about feeling trapped rather than about talking to software.
Hand off immediately whenever the customer asks for a person. Also hand off at any sign of frustration, for questions about money, cancellations, complaints or anything regulated, and after two failed attempts, because the third will not go better.
It replaces the repetitive part of the work, which is usually the part staff least enjoy. Treating it as a reason to cut staff tends to make support worse, because the conversations that remain are the difficult ones and nobody has spare time for them.
A bot that covers your top ten questions is useful from the day it goes live. Making it reliably good takes a few weeks of reading transcripts. The work on rarer questions is never finished, and beyond a certain point the remaining questions are ones a person should answer anyway.
Start small
That is a working support bot, and it takes an afternoon to set up. Expand it after you have read a week of transcripts.