AI Customer Service Agent
Build an ecommerce support agent that uses verified policies, gives short answers, and hands uncertain requests to a person.
Build an AI customer service agent around a short, approved set of product and policy facts. Let the agent look up those facts, answer in the tone customers already hear from your team, and hand a conversation to a person when the answer is missing or requires approval.
In Scott’s workshop, two versions of the same model answered the same question about course price and returns. One produced a long response with an invented price. The other used the approved price and kept the claims within the information it had. The public project includes the persona and knowledge workflow. YourAgents open-source project
What does an ecommerce support agent need to know?
Start with a small document covering current prices, availability sources, delivery, returns, support hours, and the usual questions. Put an owner and last review date on each section. Add examples of replies your team can share internally after removing customer names, order numbers, and personal details.
Give the agent specific handoff cases. Examples include changing an order, promising a refund outside policy, answering an undocumented warranty question, or resolving conflicting price sources. The reply should tell the customer what happens next.
What happens when a customer asks a question?
| Step | Action |
|---|---|
| Identify the request | Decide whether the customer is asking about a product, an order, or a policy |
| Find the source | Retrieve the relevant policy or look up the authorized record |
| Draft the answer | State the answer in ordinary language and surface any missing information |
| Check the boundary | Route out-of-scope requests to a person before a promise is made |
| Keep the context | Save the minimum necessary details in the authorized support record |
For example, “Can I return an opened item?” should lead to the returns policy for that item. If the policy covers unopened items only and has no answer for the opened case, route the conversation to support. A polite guess still creates a promise.
When do you need a retrieval system?
If the approved material fits in a short reference, start with that. When the team has many product lines, several policy versions, or years of operating documents, retrieval can bring the relevant section into the answer. Keep the source and version visible to the reviewer. Separate internal playbooks from public-facing policy and grant each channel access only to the information it needs.
How do you test the agent before customers use it?
Create a short evaluation set with common requests, missing facts, contradictory documents, and cases that should go to a person. Use examples from actual customer questions that have been cleaned of personal details. Run the same cases after edits to the prompt or policies.
Scott’s workshop caught an answer that offered a free second-week replay when the approved offer covered Week 1. That is a useful regression case because it tests a concrete fact a visitor could act on. In your evaluation set, measure whether the answer uses the right source, quotes the right price, and hands off at the right moment.
Start in a test channel. Review its first messages for factual accuracy and check that human handoff works end to end. SCT Camp Cohort 2 · AI weekly business reports