Skip to content
AskVault Docs
Try Free →

NLWeb protocol: agent-discoverable workspace endpoint

Written by , Principal Architect, Networks and Tech, AskVault · Reviewed by Allison Chan
Last updated: · 5 min read

What is NLWeb

NLWeb is an open protocol proposed by Microsoft (R.V. Guha, who also designed Schema.org and RSS) that standardizes natural-language queries on websites. The pitch: every site becomes an AI app, and both human users and AI agents query it through the same standard interface.

Each NLWeb instance is also an MCP (Model Context Protocol) server, so agents can choose between the conversational HTTP shape and the structured MCP tools shape depending on what they support best. For a structured-tools-only integration, see the per-workspace MCP server.

Why turn it on

If you do nothing, AI agents (ChatGPT browsing, Perplexity, Claude with web access) crawl your public site. They scrape rendered HTML, run their own extraction, and answer questions about your business based on whatever fragments they pulled. This can go wrong in three ways:

  1. Wrong information: extractors miss the page that actually has the answer.
  2. Stale information: they cite a cached page from weeks ago.
  3. No attribution: answers presented as the agent's own with no link back to you.

With NLWeb enabled, those same agents can query your workspace directly. They get:

  • Your canonical, indexed knowledge: the same pipeline that powers your chat widget.
  • Source citations in every response, with document name and snippet.
  • Real-time freshness: answers reflect your latest re-crawl, not a six-week-old cache.

You participate in the agentic web on your terms.

Enable NLWeb for a workspace

Turn on the endpoint from Workspace settings in three steps.

  1. Dashboard, then Workspace settings.
  2. Toggle NLWeb endpoint to on.
  3. Note your workspace ID (shown at the top of settings).

That's it. Within seconds, agents can hit:

POST https://api.askvault.co/api/nlweb/{your_workspace_id}/ask

Wire format

NLWeb defines one request shape and one response shape. Both are plain JSON over HTTP.

Request

POST /api/nlweb/{workspace_id}/ask
Content-Type: application/json


{
  "query": "Does the Starter plan include WhatsApp?",
  "site": "https://yourcompany.com",
  "locale": "en",
  "previous_responses": [
    {"role": "user", "content": "What's your pricing?"},
    {"role": "assistant", "content": "Starter is $39/mo, Growth $99/mo..."}
  ]
}

The query field takes the user's question and is the only required field. site is an optional origin reference, logged for analytics and doesn't gate the answer. locale is an ISO-639-1 code; AskVault auto-detects language from query if absent. previous_responses is optional chat history so the agent can maintain a thread across calls. See API authentication for how AskVault's own POST /v1/query endpoint compares when you control the caller yourself.

Response

HTTP/1.1 200 OK
Content-Type: application/json


{
  "answer": "Yes, the Starter plan ($39/mo) includes the WhatsApp channel...",
  "answer_type": "text",
  "sources": [
    {
      "name": "Pricing page",
      "snippet": "Starter plan includes WhatsApp Business API integration, web widget, and email assistant...",
      "relevance": 0.91
    }
  ],
  "conversation_id": "conv_abc123"
}

answer_type is "text" today. Future iterations will return "structured" when the workspace's agent picks a GenUI component (carousel, link cards, etc.).

conversation_id can be echoed back as previous_responses items on the next call to maintain a continuous thread.

Discovery endpoint

Agents that crawl your domain can also hit a dedicated discovery URL. It returns workspace metadata instead of an answer.

GET /api/nlweb/{workspace_id}

The response is a small JSON blob describing the workspace, the ask URL, and the MCP URL (if MCP is enabled). It's public and safe for agents to crawl as part of discovery.

Embedding the discovery hint on your site

Add a link tag to your site's <head> so spec-aware agents auto-discover the endpoint.

<link rel="nlweb" href="https://api.askvault.co/api/nlweb/{workspace_id}/ask">

When NLWeb-aware agents (Bing Copilot, ChatGPT browsing) crawl your site, they pick up the link and start routing future questions about your business through your workspace endpoint instead of scraping. This pairs well with an llms.txt file, which gives crawling agents a similar hint at the site level rather than the per-page level.

Rate limiting

Per (workspace, IP) token bucket: 30 requests per 60 seconds. Enough for any legitimate single-agent session; tight enough to deny a runaway bot.

Hitting the cap returns HTTP 429 with a clear message. Owner-plan monthly quota also applies; NLWeb queries count against your workspace's chat quota the same way API queries do.

Security model

  • Public read: no auth on the request. NLWeb's whole point is to be agent-discoverable.
  • Owner opt-in: NLWeb is off by default. Nothing surfaces until you explicitly enable it.
  • Plan-quota gated: abusive query volume burns the owner's plan quota, same as the chat widget.
  • No mutating tools: the NLWeb surface is read-only. Skills that mutate state (collect_lead, escalate_to_human) require the in-widget confirmation flow and don't run on NLWeb calls.

What's NOT in v1

  • Streaming responses: v1 returns the complete answer in one HTTP response. Streaming arrives when the spec's streaming variant stabilizes.
  • Structured (GenUI) answers: answer_type is always "text" today; carousel/card responses ship next.
  • Per-end-user identity verification: every NLWeb caller is treated as anonymous. Agent-side identity (the actual end user behind the agent) is the agent's responsibility to surface.

See also

  • Per-workspace MCP server: same idea, structured tool shape for Claude Desktop, Cursor, and ChatGPT desktop.
  • Query decontextualization: multi-turn retrieval quality fix that runs on NLWeb calls too.
  • POST /v1/query: the authenticated API endpoint for callers you control directly, versus NLWeb's public agent-facing surface.
  • Rate limits per plan: how the 30 requests per 60 seconds NLWeb cap compares to your plan's API rate limit.
Was this page helpful?