Skip to content
AskVault Docs
Try Free →

The Chat Playground

Written by , Principal Architect, Networks and Tech, AskVault · Reviewed by Allison Chan
Last updated: · 3 min read

What the Playground is for

Two patterns:

  1. Answer-quality testing. Ask sample questions before launching a bot or after changing knowledge content, and see exactly which sources it retrieved.
  2. Prompt tuning. Edit the workspace's draft system prompt in Settings and see its effect immediately — nothing changes for real visitors until you explicitly save it.

Playground messages run through the same pipeline as a real conversation and count toward your monthly message quota — they aren't free or unlimited.

How to use it

  1. Open Chat Playground from the dashboard sidebar.
  2. Type a question in the chat input.
  3. Bot responds, retrieving from your real knowledge base.
  4. Click the response to open its Answer Insight panel:
    • Retrieved sources, each with a relevance score.
    • Tokens used.
    • Total response latency.

Compare models side-by-side

  1. Switch to Compare mode.
  2. Pick a left model and a right model from your available roster (models your plan doesn't grant show as locked, not selectable).
  3. Send a question. Both models answer the same query, run against the same retrieval pass.
  4. See both answers side-by-side, each with its own sources and metrics.

Compare mode doesn't save either message to conversation history, but each comparison still counts toward your message quota. Useful for choosing a default model or debugging a quality difference between two candidates.

Retrieved sources

For every answer, the Answer Insight panel lists which chunks fed the response:

  • Document name.
  • Relevance score, shown as a percentage.

If you see a low-relevance source at the top of the list, that's a retrieval problem, not an LLM problem — tune chunking, add a snippet, or refine the source content.

Tuning the system prompt

Under the Playground's Settings panel you can edit:

  • System prompt. Loaded from the workspace's draft prompt, not the live one — editing and testing here never affects real visitor conversations until you explicitly save.
  • Temperature, top-K, and strict mode. RAG and generation knobs, applied to Playground messages as you test.
  • Branding and agent name (branding removal requires Growth+ or the branding add-on).

Saving publishes your changes to the live workspace — there's a confirmation step before that live write goes through.

Sample test scenarios

Patterns worth running before launch:

Happy path. Common customer questions ("how much does it cost?", "what's your refund policy?"). Check the answer and its sources.

Edge cases. Questions outside the knowledge base. Confirm the bot doesn't hallucinate and says it doesn't know, cleanly.

Multi-turn. A short back-and-forth. Verify context carries across turns.

Common pitfalls

Playground bot behaves differently from production. You're testing an unsaved draft prompt. Save it under Settings to make it live, or compare the draft against the live prompt directly.

Slow first response. A cold workspace with a vector index that hasn't warmed up. The next query is typically faster.

Citations don't show. Source content isn't indexed yet. Check the Knowledge Hub for indexing status.

Playground messages using up your quota faster than expected. Expected — Playground messages count the same as any other channel. Budget for it when testing heavily near a plan's limit.

Planned features (on the roadmap)

Documented for accuracy — none of the following exist yet:

  • A quota-exempt test mode. Today, every Playground message counts against your monthly quota. A free/unmetered testing lane is not built.
  • A skill-trace panel. Today, skill firing shows up the same way it does in a real conversation (e.g. a GenUI card), with no dedicated trigger/policy-check trace view.
  • Visitor identity simulation and channel-format simulation (previewing a widget answer reformatted for WhatsApp/SMS/voice).
  • Saved, shareable Playground sessions with an audit/rollback log.
  • Three-way (or more) model comparison. Compare mode is two models today.

FAQ

Does the playground use real customer data?

No — Playground conversations are your own test messages and aren't stored in Live Chat. They do retrieve from your real, live knowledge base.

Will my playground prompts leak to production?

Only once you explicitly save. The Playground edits the workspace's draft system prompt; the live prompt used for real visitors only changes when you save.

Does the playground count against my message quota?

Yes. A Playground message — including each side of a model comparison — runs through the same chat pipeline as production and is checked against your plan's quota.

Was this page helpful?