“How do I change the delivery address?”
The bot explains where to find the right section and sends a link to the instructions.
“My order no longer lets me change the address.”
The bot suggests opening the order settings again.
“Let me speak to a person.”
By the time a support agent joins, the customer has already spent time without resolving the issue. Now they need to repeat the question, provide the order number, and explain again why the instructions did not help.
The bot technically responded quickly. But support still needs to handle the request, and the customer has been through several extra steps.
To reduce the support workload, measure whether automation helped resolve the issue and how much effort it required from the customer.
We will examine why conversations stall, which actions AI can handle, and how to test the benefit in one scenario before expanding the project.
The customer needs an outcome, but the bot provides instructions
Some inquiries really can be resolved with reference information.
A person asks about opening hours, payment methods, or delivery terms. If the answer is current and clear, the conversation may end there.
In other cases, the customer wants to change a specific situation:
- find out where their order is;
- reschedule delivery;
- correct contact details;
- register a return request;
- restore account access;
- resolve a duplicate charge.
General instructions may not be enough.
To help, the system needs data about the specific order or account, available actions, and rules for performing them. A bot with only a collection of answers can explain a procedure but may not be able to guide the customer through it.
The problem arises when that limitation is hidden. The customer expects a resolution while the bot keeps suggesting similar articles.
Before automating each scenario, complete this sentence: “By the end of this conversation, the customer should have…”
For example:
By the end of this conversation, the customer should have confirmation of a new delivery address or a registered request for an employee to check whether the change is possible.
That is a goal you can test.
Why asking for a person does not automatically mean failure
Sometimes involving an employee is the right outcome.
The customer may dispute a transaction, face an unusual situation, or simply want to speak to a person. They should not have to prove that the bot has failed.
Distinguish between two scenarios.
In the first, the bot understands the request, collects the necessary information, and passes it to the responsible person. The employee continues where the conversation left off.
In the second, the customer rephrases the question several times, receives identical answers, and only then reaches a support agent.
Both involve a human handoff. But the time spent and the customer's experience are different.
A goal of involving agents as rarely as possible can therefore create a frustrating process. It is more useful to define which issues the system should resolve independently and when it must promptly hand work to an employee.
Find where the conversation stops helping
Start with inquiries where customers asked for an agent after speaking to the bot.
For an initial review, use a small sample from a typical working period. Include different topics and outcomes. This can reveal recurring problems but does not replace assessment of the full flow.
Complete a table for each conversation:
Audit of conversations where customers request a human agent| What to check | What to record |
|---|
| Customer's task | The outcome they wanted |
|---|
| Bot response | The information or action the system offered |
|---|
| Obstacle | Why the customer could not proceed |
|---|
| Available data | Whether the bot could see the relevant order, status, or history |
|---|
| Ability to act | Whether the bot could perform the operation or create a ticket |
|---|
| Reason for handoff | System limitation, error, exception, or customer request |
|---|
| Repeated information | Whether the customer had to explain the situation again |
|---|
| Outcome | Resolved, assigned for handling, or still open |
|---|
Look for a specific cause.
“Customers dislike the bot” offers little guidance for improvement. “The bot suggests changing the address themselves even though the order has already been dispatched” identifies a concrete problem: the system ignores order status.
Four reasons customers get stuck
1. The bot cannot see data about the specific request
It knows general delivery times but not what is happening to the customer's order.
In this case, it needs access to a current data source. The system must also check whether the user is entitled to receive information about that order. Knowing the order number alone may not be sufficient.
If the source is temporarily unavailable, the bot should explain that and offer a workable next step. Inventing a status will only complicate the case.
2. The bot can explain an action but cannot perform it
It explains how to request a return, but the customer still needs to find a form, re-enter information, and attach photographs.
Part of this process can be shortened: collect the information in the conversation and create a draft ticket or submit the request if company rules allow it.
Distinguish the outcomes. “Your return request has been registered” and “Your return has been approved” mean different things. The system must accurately state what happened.
3. The bot does not notice that its previous answer failed to help
The customer says, “I already tried that.” The system repeats the same instructions in different words.
This needs a separate path: clarify the specific obstacle, offer another permitted method, or involve an employee.
Repeating help text indefinitely does not resolve a request.
4. The human handoff resets the conversation
The agent sees only the latest message or a brief “needs help” note.
The customer repeats the order number, problem description, and actions already taken. Information may be lost or altered in the retelling.
Give the employee the original conversation along with a short summary: the customer's task, verified data, actions taken, and unresolved question. An AI summary must not replace access to the original exchange.
Which actions can AI handle?
You do not need to build an assistant that resolves every request independently from day one.
Choose a recurring scenario and define its boundaries.
AI assistant actions and human handoff conditions| Scenario | What the assistant can do | When an employee is needed |
|---|
| Order status check | Retrieve current status after checking access and explain the next step | Data conflicts or the status requires investigation |
|---|
| Address change | Check eligibility, show the new address for confirmation, and perform the permitted operation | The order has reached a stage requiring manual approval |
|---|
| Return request | Collect required information and register the request | A decision requires product inspection or exceptional terms |
|---|
| Access recovery | Guide the user through the approved recovery process | Standard verification fails or access is disputed |
|---|
| Payment complaint | Collect information and route the request to the right specialist | A financial decision or investigation is required |
|---|
Capabilities depend on your systems and rules. This table is a design starting point, not blanket authorization for the listed actions.
AI can help interpret free-form requests and conduct the conversation. Access checks, operation eligibility, and mandatory confirmations must be enforced by software rules.
For example, an address change should not happen just because the model decides that it seems allowed.
What a useful handoff looks like
Return to the address change example.
The customer says editing is unavailable in their account. The assistant checks the status and finds that the order has already been dispatched.
It can explain the limitation and, if the procedure allows, collect the new address and create a request for a specialist.
The customer receives:
- confirmation that the request has been registered;
- a ticket number;
- information about the next step;
- a response deadline, if the company can actually meet it.
The agent receives:
- the identified order;
- its current status;
- the new address supplied by the customer;
- details of completed checks;
- the question requiring a decision;
- the original conversation.
The address has not changed yet. But progress has been made, and the customer does not have to start over.
This handoff can be more valuable than a long conversation that ends without either an agent or a resolution.
What to measure instead of bot response counts
Message volume and first response speed show system activity. Assessing value requires metrics connected to outcomes.
Metrics for assessing support automation| Metric | What it tells you |
|---|
| Verified resolution rate | How many tasks were actually completed |
|---|
| Repeat contacts about the same issue | Whether customers return because the solution did not help |
|---|
| Time to resolution | Whether customers get an outcome faster |
|---|
| Employee time per request | Whether manual work has decreased |
|---|
| Handoff quality | Whether employees receive the data needed to continue |
|---|
| Incorrect answers and actions | Whether automation creates new problems |
|---|
| Customer effort | Whether people repeat information and go through extra steps |
|---|
Define what “resolved” means for each scenario in advance.
For an address change, evidence may be a successful operation in the system and a customer notification. An informational question needs a different approach, such as sampled conversation reviews and feedback.
Do not treat customer silence as proof of success. They may have received an answer, become tired of the conversation, or gone elsewhere for help.
Review repeat contacts in context too. A new question about the same order does not always mean the previous issue remained unresolved.
How to evaluate the economics of one scenario
Suppose a company receives 300 requests of a particular type each month. Previously, an employee spent an average of six minutes on each.
The pilot finds that:
- the assistant correctly resolves 180 requests without an agent;
- 120 requests are handed to an employee;
- handling a transferred request takes an average of four minutes because the necessary information has already been collected.
Before automation, this group of requests required:
300 × 6 = 1,800 minutes, or 30 hours.
After automation, agents spend this much time handling transferred requests:
120 × 4 = 480 minutes, or 8 hours.
The preliminary difference is 22 hours per month. Subtract time spent on quality control, corrections, and maintaining the workflow. Repeat contacts caused by errors must also be included.
This is a hypothetical example, not a promised result. Freed-up time does not automatically reduce payroll.
Practical benefits might include a shorter queue, the ability to handle more requests, or less overtime. Evaluate financial impact based on changes that actually occur, accounting for implementation and operating costs.
Where to start if your bot is already running
You do not necessarily need to rebuild it.
Choose one common reason for human escalation. Identify the step the bot cannot complete and decide what to add: current data, an authorized action, a different clarification, or a context handoff.
Record baseline metrics before making changes. After the improvement, test ordinary inquiries, exceptions, external system outages, and repeated requests. A repeated request must not cause the same operation to execute twice.
Initially, uncertain actions can remain subject to employee confirmation. Expand the assistant's autonomy after checking the quality of the specific scenario.
At Futureinapps, we build AI customer assistants and connect them to company systems. Start with a few anonymized conversations where the bot failed to help and a description of the outcome the customer should have received.
This can help define the first scenario and how to test its value.
Good support automation shortens the customer's path to resolution. Sometimes that requires an accurate answer, sometimes a system action, and sometimes timely involvement of a person.
