Improve response quality
Diagnose weak answers by testing sources, instructions, and models separately in Playground.
You improve response quality faster when you know which layer failed. A weak answer can come from missing knowledge, an ambiguous instruction, an unsuitable model, or a Playground candidate tested but never saved to the agent. Playground gives you controls to test these causes without changing everything at once.
Start with an acceptance set
Before you test, write down the expected result. A useful starter set contains:
| Test | What a passing response does |
|---|---|
| Known fact | Returns the exact policy, number, or step supported by a source |
| Paraphrase | Finds the same fact when the customer's wording differs |
| Ambiguous request | Asks for the information needed to answer safely |
| Unsupported request | Admits the knowledge is insufficient instead of inventing an answer |
| Boundary test | Refuses or redirects a request outside the agent's defined job |
| Multi-part question | Answers each supported part and separates any uncertainty |
Keep these prompts stable while you tune the agent. A moving acceptance set makes before-and-after comparisons unreliable.
Test prompt changes without saving them
Open the agent's Playground. Edit System Prompt, or choose another Model. Then select Apply & Test. Playground recreates the chat with these overrides. You can then test the candidate configuration.
The Unsaved changes badge means the editor differs from the agent's stored configuration. Apply & Test does not persist those changes. After the candidate passes your acceptance set, select Save to Agent. Select Reset to return the editor to the last saved model and prompt.
This difference matters when a live widget behaves differently from a successful Playground test. First, confirm that you saved the winning configuration to the agent.
Diagnose the failure before changing the model
Use this order:
- Confirm the source. Confirm that processing is complete, the relevant page is indexed, and its text contains the expected fact.
- Remove conflicts. Look for older pages or Q&A entries that give a different answer.
- Clarify the system prompt. State the boundary or response behavior directly. Do not bury it in repeated prose.
- Retest the exact prompt. Confirm that the change fixed the original failure and did not break the other acceptance cases.
- Compare models. When the knowledge and instructions are already sound, escalate the model.
A model cannot reliably recover a fact that never entered the knowledge base. But more source text does not fix an instruction that rewards confident guessing.
Compare candidate configurations
Select Compare Models in Playground to open synchronized panels. Comparison mode starts with multiple panels and supports up to four. Each active panel can have its own model, temperature, and system prompt. A shared input sends the same message to every active panel. The header shows the combined credit cost before you send.
Every panel shares the agent's saved sources and retrieval settings. Comparison mode varies only the model, prompt, and temperature, not the knowledge configuration. It also does not show the source chips available in normal Playground mode. When you need to inspect which sources supported an answer, use normal mode.
For a fair comparison:
- Keep the system prompt and temperature equal when you compare models.
- Keep the model equal when you compare prompt variants.
- Use the same prompts in the same order.
- Grade factual correctness before style.
- Record which failures each candidate makes.
- Repeat important tests because model output can vary.
When one panel wins across the acceptance set, use that panel's Save to Agent control. This saves its model and system prompt. Temperature remains a comparison-only setting. Do not choose based on a single unusually good response.
Write instructions you can test
Prefer observable rules. For example, you can test the rule "Use no more than five sentences unless the customer requests detail." You cannot test "Be excellent." Tell the agent how to handle missing knowledge. Do not script a false answer for every possible gap.
A practical system prompt usually covers role, audience, source discipline, response format, uncertainty, and escalation. Keep volatile product facts out of it. Those facts belong in sources, so that their owner can update them independently.
Confirm the deployed experience
After you save a configuration, repeat the acceptance set on the actual channel. For the website widget, confirm that the agent is Public. Also confirm that the current domain is allowed and that the embed code is installed. Use Activity to confirm that the live conversation reached the expected agent.
If you need a starting prompt and deployment sequence, return to Create your first agent. For durable operating habits, see Agent best practices.