Skip to content
AskVault Docs
Try Free →

AI Insights, the improvement loop

Written by , Principal Architect, Networks and Tech, AskVault · Reviewed by Allison Chan
Last updated: · 5 min read

The loop, in one picture

A support agent is only as good as the gaps you keep closing. AI Insights is the closed loop:

  1. Answer. The agent handles conversations across your channels.
  2. Measure. A resolution-rate KPI tells you what fraction it actually resolved.
  3. Find the gaps. An analysis pass clusters the failures into a short list of topics and tells you why each one failed.
  4. Fix. For each cluster it drafts an answer; you approve it and it becomes trained knowledge.
  5. Repeat. The next time those questions come in, the agent answers them.

It lives in the Analytics page, below the standard charts. Growth+ Growth plans and above include it, with up to 5 workspaces and a 40 MB knowledge base before you need Business or Enterprise.

Resolution rate

The headline number. Most chatbot dashboards show you "queries" and "satisfaction." Resolution rate is the metric buyers actually care about: did the conversation get handled without a human.

AskVault computes it automatically from outcomes already recorded on each answer, with no extra tagging from you:

  • Resolved = the agent answered the question AND the visitor did not thumb it down.
  • Unresolved = the agent could not answer, OR the visitor thumbed the answer down.

Resolution rate is resolved / total over the date range. It uses no LLM call, so it loads instantly alongside the rest of your analytics. Track it week over week; the whole point of the loop below is to push this number up.

AI Recommendations

This is the on-demand part. Click Analyze failed questions and AskVault pulls the questions behind your failed conversations in the window, where "failed" means the agent did not answer it, the visitor thumbed it down, or retrieval confidence was very low (a likely-bad answer the visitor never rated).

It then clusters those questions into a short list of topics (up to 6), and for each cluster returns:

  • A topic label. Plain-language summary of what this group of questions is about.
  • Example questions. The real visitor questions in the cluster.
  • A count. How many failed questions fell into this topic, so you fix the biggest gaps first.
  • A failure reason. One of four diagnoses:
    • Content missing - the answer simply is not in your knowledge base. Add it.
    • Unclear - the content exists but is ambiguous or buried. Rewrite or add a direct answer.
    • Contradictory - two sources disagree, so the agent could not commit. Reconcile them.
    • Out of scope - the question is outside what this agent should handle. No fix needed; consider an escalation rule instead.
  • A draft answer. A suggested answer grounded in a sample of your existing knowledge, ready to edit.

The analysis runs on a fast, low-cost model that is separate from the model answering your customers, so it does not consume your chat message allowance. If the analysis model is unavailable for any reason, the panel degrades to a simple text-grouping fallback so it still renders. It never blocks the rest of the dashboard.

One click to close the gap

Every recommendation has an Apply as Q&A button. Click it and the drafted answer is saved as a trained Question-and-Answer pair in your knowledge base. Q&A pairs get the highest retrieval priority, so the next time a visitor asks anything in that cluster, the agent answers from your approved text.

That is the loop closing in one click: a failed conversation becomes a measured gap becomes a drafted fix becomes trained knowledge, without leaving the Analytics page. Edit the draft before applying it whenever the wording matters.

How to use it well

  • Run it weekly. New questions arrive faster than you can guess at them. A weekly pass over the last 7 days keeps the unresolved list from compounding.
  • Fix by count, not by order. The cluster counts tell you which gap is costing you the most conversations. Start at the top.
  • Treat "out of scope" differently. Those are not knowledge gaps. If a topic genuinely is not this agent's job, set up an escalate-to-human path instead of forcing an answer.
  • Pair it with the Unanswered Queries panel. The Analytics page lists raw unanswered questions; AI Insights is the layer that groups and diagnoses them. Use the panel to spot, use Insights to fix.

FAQ

Which plans include AI Insights?

The resolution-rate KPI and the AI Recommendations analysis are available on Growth and above. On lower plans the section shows an upgrade prompt.

Does running the analysis use my message quota?

No. The clustering and diagnosis run on a separate fast analysis model, not the chat model answering your customers, so it does not draw down your monthly chat messages.

How is "resolved" decided if the visitor never left feedback?

An answered turn counts as resolved unless the visitor explicitly thumbed it down. Most visitors do not rate answers, so the resolution rate reflects "the agent answered and the visitor did not complain," which is the realistic signal at support volume.

Can I edit a drafted answer before adding it?

Yes. The draft is a starting point. Edit it for tone and accuracy, then apply it as a Q&A pair.

What is the difference between this and Unanswered Queries?

Unanswered Queries (in Analytics) is the raw list of questions retrieval came back weak on. AI Insights groups those into topics, diagnoses why each failed, and drafts a fix you can apply in one click.

Was this page helpful?