Containment is not resolution, and optimising the first destroys the second

An AI agent goes live in support, and within a month the dashboard shows seven in ten conversations "handled without a human". Meanwhile phone volume has not fallen, complaints mention "the bot", and customers are writing in twice about the same order. Nothing is broken in the reporting; it is measuring the wrong thing. This piece explains the gap, the metric that closes it, and how to design an agent that is judged by whether it helped.

• Deflection: the contact never reached a queue.
• Containment: the interaction ended without escalation to a human.
• Resolution: the customer's problem was actually solved.

A system can score well on the first two while failing the third, and the failure is invisible in a dashboard that reports only those two.

Containment is measured at the end of an interaction. Resolution can only be measured afterwards: did they come back, did the underlying thing get fixed?

Anything measured at the end of an interaction can be improved by making the interaction harder to leave. A customer who gives up on the chat and phones instead is "contained" by every definition the dashboard uses, and has cost you two contacts instead of one. Nobody chose this metric; it accumulated, because "did this session reach a human?" is the easiest thing to instrument.

Generative AI agents add a second risk. They rarely get stuck; they answer. A confident wrong answer ends the conversation — counted as containment — and creates a problem that returns days later, often on a different channel.

Published figures for containment and resolution vary widely between sources, which itself shows the definitions are not stable. Treat any benchmark with caution; the durable point is that the gap exists and is often large.

Resolution rate: the share of conversations where the customer did not come back about the same issue, on any channel, within a defined window.

• The window. Seven days suits most consumer support; thirty for anything tied to a delivery or billing cycle. State which you used; the number is meaningless without it.
• Same issue, not same customer. A customer returning about something new is a good sign. You need issue-level identity — usually a topic classifier over the conversation text, not a category someone picked.
• Count channel switches. A contained chat followed by a call about the same thing is a failure, and the most expensive kind. If your chat and voice systems cannot see each other, the metric will flatter you.

A retailer's agent handles 10,000 conversations in a month.

Of the 7,100 contained conversations, 2,800 customers came back within seven days about the same issue — 1,900 by chat again, 900 by phone. So 4,300 conversations, 43% of the total, were truly resolved by the agent. The team then changed three things: escalation offered after two failed attempts, full transcript passed to the human, and an explicit "Did that solve it?" at the end. Containment fell to 64%; resolution rose to 52%; phone contacts about chat issues fell by a third.

Two more design rules:

• Tell people it is an AI. It is required in some places (see AI Act Article 50) and it changes behaviour for the better: people ask clearer questions and escalate rather than argue.
• Ground the answers. Limit the agent to your own policies, order data and help content, and have it hand over rather than guess when those do not cover the question.

If the objective is cost reduction, say so and measure cost per resolved issue. That is a legitimate goal and it keeps the incentive attached to resolution. Cost per contained contact optimises a number that can rise while the business gets worse. The failure mode is claiming a service improvement while measuring a cost one: it works for two quarters, until someone compares the containment chart with complaint volume.

• How do you define containment and resolution, and can I change the return window?
• Can you detect a customer returning on another channel — including voice — about the same issue?
• What exactly does the human see at handover: the full transcript, the customer's identity, the agent's reasoning?
• What does the agent do when your knowledge sources do not cover a question?

Next in this track: measuring a conversational programme. For the disclosure rules that now apply to AI agents in the EU, read AI Act Article 50 for messaging; for the queue the agent hands over to, see the support queue as a system.