Query decontextualization for multi-turn chat
The problem
Follow-up questions break naive vector search because they lean on words spoken earlier in the chat. A user opens your chat widget and types two messages:
Turn 1: "What's your pricing?"Bot: "Starter is $39/mo, Growth $99/mo..."Turn 2: "Does it include WhatsApp?"The vector embedding of the second message has no signal about what "it" refers to. Searching the vector index for "Does it include WhatsApp?" retrieves chunks about WhatsApp generally (number setup, message templates, API docs), not chunks about your pricing plans.
Result: the LLM answers correctly only by luck (because the conversation history is in its context), or it cites the wrong page, or it says "I don't see WhatsApp in our pricing" because the relevant pricing chunk never made it into the top-K.
This is the single largest accuracy bug in naive RAG pipelines.
How AskVault fixes it
AskVault rewrites the user's message into a standalone query before the vector search runs, using the last few turns as context. See how vector databases work for background on what that search step is actually doing.
Original: "does it include WhatsApp?"Decontextualized: "Does AskVault's Starter Plan pricing include WhatsApp?"The rewritten query is what hits the vector index. The pricing chunks come back. The agent answers correctly with the right source citation, whether the visitor is comparing the Starter Plan, the Growth Plan, the Business Plan, or the custom-priced Enterprise Plan.
When the rewrite happens (and when it skips)
| Condition | Rewriter behavior |
|---|---|
| First turn of a conversation | Skipped; message is already standalone |
| Message is a greeting/thanks ("hi", "ok", "thanks") | Skipped via heuristic |
| Message is one or two words | Skipped; no referential signal to resolve |
| Multi-turn follow-up with prior history | Rewritten using a fast model |
It runs automatically on every workspace; there is nothing to configure.
What model runs the rewrite
A fast, lightweight model, not your workspace's configured chat model, folds in the last few turns of history. This means:
- The rewrite call is not metered against your customer's monthly message allowance.
- It adds only a small amount of latency, well inside the perception threshold, and stays well under 1 second even on a slow connection.
- On any failure (timeout, parse error, network), the original query is passed through unchanged. The chat turn never fails because of decontextualization.
What you'll see in your data
Retrieval precision on follow-up turns rises when a workspace has active multi-turn conversations. Published RAG benchmarks put the gain at roughly 15 to 30 points, and the exact lift varies by knowledge base shape. Along with that:
- Source citation accuracy improves: the agent attributes answers to the page that actually contained the answer instead of an adjacent page that happened to be lexically close. This ties into how reranking orders the retrieved chunks before the answer is written.
- Latency increases slightly on follow-up turns only. First turns are unaffected, since there's no history to fold in.
Configuration
There is nothing to configure. Query decontextualization runs automatically on every workspace, and is skipped on first turns, greetings, and very short messages (see the table above). It folds in a small window of recent turns as context, tuned so it resolves references like "it" and "that plan" without over-compressing the conversation. It runs the same way whether you're on the 14 day free trial or a paid plan running dozens of workspaces. The pricing plan comparison shows message allowances by plan, since every rewrite call and every answer draw from the same monthly message pool. You can check exact usage against the conversation history API if you want to see decontextualized queries alongside raw user turns.
Why this matters as chat gets more conversational
As chat surfaces shift from one-shot Q&A to genuine multi-turn dialogue, the percentage of queries that are follow-ups climbs. A pipeline that's 90% accurate on first-turn queries but 60% accurate on follow-ups feels broken to users.
Query decontextualization is one of the low-cost RAG fixes that a serious chat system runs by default. AskVault has it on for every plan, including the 14-day free trial; competitors like Chatbase and SiteGPT pass the raw query to vector search and rely on the LLM to recover, which works some of the time. Pass the conversation_id from each response back into the next query request so AskVault has the prior turns available to fold into the rewrite.