You've met the bad version. You ask a support bot a simple question, it replies with three paragraphs of generic advice, and there's no way to reach a human. The model behind it may be excellent. The design around it failed.
When we build support assistants for clients, most of the effort goes into the parts around the model.
Give it a narrow job
We start by listing the questions customers ask most, using the client's support inbox or ticket history. Usually a short list covers a large share of volume: order status, returns, password resets, delivery times, pricing tiers.
The assistant handles those well and says so when a question falls outside them. A bot that tries to answer everything will answer some things wrong, and customers remember the wrong answers.
Answer from your own content
The assistant should answer from the business's own help articles, policies and product data, not from what the model learned on the internet. We connect it to those sources through retrieval: find the relevant passages for each question, then ask the model to answer using only those passages.
Two rules keep this honest:
- If retrieval finds nothing relevant, the assistant says it doesn't know and offers a human.
- Each answer links to the help article it came from, so the customer can check.
Connect it to real data, carefully
"Where is my order?" needs the order system, not a help article. We give the assistant a small number of read-only tools: look up an order by number for the signed-in customer, check stock, find the nearest store. Each tool checks permissions on the server. The model never gets a raw database connection, and it can't issue refunds or change accounts on its own.
Make the handoff fast
The escape hatch matters more than the bot. Customers should reach a person in one click, and the agent should see the conversation so far. We also hand off automatically when the customer repeats a question, uses words like "complaint" or "cancel", or when the assistant's confidence is low.
Measure it like a support channel
We track the numbers a support manager already cares about:
- Share of conversations resolved without a human
- Handoff rate, and what the handed-off questions were about
- Customer rating at the end of the chat
- Wrong answers found in weekly review
That last one needs a person. Each week someone reads a sample of conversations and flags bad answers. Those become test cases, so the same mistake gets caught before the next release.
Where it goes wrong
Most failed assistants we've been asked to fix had the same problems: a vague system prompt, no retrieval, no handoff and no review. Swapping to a newer model helped less than fixing those four things.
If you're planning a support assistant, start with your ticket history rather than a model comparison. The history tells you what to build.
