Improvements: learn from conversations that went badly

The agent reads its own difficult conversations and suggests changes to its prompt. You test and approve each one.

A review reads the conversations where staff stepped in, the caller rated it bad, or it ended unresolved or unhappy. It looks for problems that keep coming up and suggests small changes to the prompt. Nothing changes until you apply a suggestion.

Get suggestions

1. Open the agent and go to Improvements. Agents that follow procedures instead of a prompt can't use it yet.
2. Click Review now, or switch on Review every week.
3. Each suggestion says what goes wrong, links to the conversations it came from, and shows the words it would add or replace.

Test and apply

1. Read the wording and change it if you like. Where the AI doesn't know a fact, it writes something like [owner: delivery charge]. Replace that with your real details.
2. Click Run tests. This runs the suggestion's own test and up to 20 of the agent's saved tests on the changed prompt.
3. When they pass, click Apply, or Try on 20% first to run it as an A/B test. If it isn't right, click Dismiss and it won't be suggested again.

• A suggestion can only be applied after this exact change has passed. If you edit it or the prompt changes, test it again.
• A saved test that fails because of the change blocks it. A test that was already failing without the change does not.
• Applying saves a new version, so you can restore the old prompt, and keeps the suggestion's test with the agent's saved tests.

What keeps it safe

• A suggestion must come from at least two different callers, so one person can't talk the agent into a new rule.
• Phone numbers, emails, and card, Aadhaar and PAN numbers are masked before the AI reads a conversation. Suggestions that contain them are dropped. Names can't be masked this way, so check the wording before you apply it.
• Deleting a conversation also removes any suggestion waiting on it.

What it costs

Reviews and test runs use the agent's own AI model and are charged like the test chat. With fewer than three new conversations to learn from, a review stops without using the AI, so it costs nothing. Weekly reviews are off until you switch them on.

Related guides