Module 6 of 6 · Intermediate · 10 min read

Customer support bots

Build a support assistant that answers from your help content, hands off gracefully and never makes up policy.

Key takeaways

1Ground the bot in your own help content, and make 'I don't know' plus handoff the default for gaps.
2Write a system prompt with scope, tone, escalation rules and forbidden promises.
3Test on real past tickets before launch, and review transcripts every week.
4Be clear with customers that they are talking to an AI, and make reaching a person easy.

01Where AI helps

A good bot handles the repetitive questions and makes the hard ones faster for your team.

Answering common questions instantly, at any hour.
Collecting details (order number, device, steps tried) before handing to a person.
Drafting replies for human agents to approve and send.
Summarising long threads and tagging tickets for reporting.
Finding gaps in your help centre from the questions it can't answer.

02The workflow

See RAG and grounding, and Testing prompts, for the techniques behind steps 4 and 5.

Collect and clean your help content: articles, policies, macros, product docs.
Pull a few hundred real past tickets and group them by type.
Write the system prompt: scope, tone, escalation rules and boundaries.
Connect retrieval over your help content, and tools such as order lookup if needed.
Evaluate on past tickets with a rubric, fix failures, repeat.
Launch as agent-assist or to a small share of traffic, then expand.

03Tools to reach for

Start with your help-desk's built-in option if it has one; build custom only when you need tools or control it lacks.

Help-desk platforms with built-in AI agents trained on your help centre.
Chat widget builders that connect a model to your documents.
Custom builds with a model API, retrieval and your own tools for full control.
Analytics on resolution, handoff rate and customer satisfaction.

04Prompt: the system prompt

Make the boundaries concrete. Most support-bot failures are promises it shouldn't have made.

Example

You are the AI support assistant for {{company}}. You help customers with {{scope}} using only the help articles in <articles> and the results of your tools. Tone: warm, brief and plain. Most customers are on their phones. Rules: - If the answer isn't in the articles or tools, say you don't know and offer a human agent. - Never promise refunds, credits, delivery dates or exceptions. Explain the policy and offer a human for exceptions. - Hand off immediately if the customer asks for a person, is very upset, mentions legal action, safety or a security issue. - Don't ask for passwords or full card numbers. - Tell customers you are an AI assistant if they ask. When you hand off, summarise the issue and what you've tried in two lines for the agent.

05Prompt: evaluate on past tickets

Grade the bot's answers to real questions before customers see them.

Example

Grade the bot reply below against the help articles. Ticket: <ticket>{{customer message}}</ticket> Reply: <reply>{{bot reply}}</reply> Articles: <articles>{{articles}}</articles> PASS or FAIL for each: 1) answers the actual question, 2) every claim is supported by the articles, 3) makes no promise outside policy, 4) hands off when it should, 5) tone is friendly and concise. Quote the evidence for each FAIL.

06Prompt: find help-centre gaps

Unanswered questions are a roadmap for your documentation.

Example

Here are {{n}} conversations where the bot handed off or said it didn't know: <conversations>{{export}}</conversations> Group them into topics. For each topic: how many conversations, a typical customer question, and the help article we should write or fix, with a suggested title and outline.

07Pitfalls

Measure resolved conversations and customer satisfaction, not just deflection. A bot that deflects without resolving just moves the work.

Invented policy: the most expensive failure. Ground strictly and forbid promises.
Handoff walls: customers who can't reach a person get angrier. Make escalation easy.
Stale content: the bot is only as current as your help centre. Assign an owner to keep it updated.
Prompt injection: customers can try to manipulate the bot. Limit its tools and never let it change accounts without verification.
No review: read a sample of transcripts every week; metrics alone hide bad answers.

Watch out

In many places, consumer-protection and AI rules may require you to disclose automated agents and handle personal data carefully. Check the rules that apply to your business.

Finished reading?

Mark it complete to track your progress, then test yourself with the Build with AI: Software & Media quiz.

Take the Build with AI: Software & Media quiz →