Response feedback

Every reply your agent gives carries a quiet thumbs-up / thumbs-down under it in the chat widget. Ratings land on the agent's Response feedback page at /app/agents/{agent}/feedback, where a thumbs-down is broken down by what kind of wrong it was — which is what tells you whether to fix the knowledge base or the crawl.

This is the visitor-driven half of the self-improvement loop. Content gaps catch the questions your agent knew it couldn't answer; response feedback catches the ones it answered confidently and wrongly — the failures no automatic detector can see.

What the visitor sees

• Two muted thumb icons appear under each finished assistant reply. They never appear on a reply that is still streaming, on an error bubble, or on a message typed by a human operator.
• Thumbs up submits immediately and collapses to a short "Thanks for your feedback!" line.
• Thumbs down also submits immediately, and then opens a small form: four reason chips (Not correct, Not what I asked, Incomplete, Something else) plus an optional free-text box.

The order matters. Most visitors abandon a feedback form, so the negative rating is recorded before we ask why — a rating you hold beats a reason you never get. Each later interaction (picking a chip, sending a comment) updates the same record rather than creating another one.

The endpoint

POST /api/v1/widget/messages/feedback, authenticated by the widget JWT like every other visitor endpoint, rate-limited to 120 requests per minute per visitor session.

{
  "message_id": "0198e4...",     // the assistant message being rated
  "rating": "negative",           // positive | negative
  "reason": "incorrect",          // optional: incorrect | irrelevant | incomplete | other
  "comment": "It only goes to 46.",  // optional, max 2000 chars
  "page_url": "https://example.com/boston"
}

The conversation is read from the signed token, never from the request body, and the message must belong to that conversation — so a visitor can only ever rate a reply from their own thread. An unknown message id returns 404 message_not_found.

It is not on the hot path. The rating is its own request; nothing in the SSE stream waits on it, and a failed POST is swallowed by the widget rather than surfaced as an error.

Where the message id comes from

The stream's start event already carries the server's message_id, emitted before the first token. The widget binds it to the bubble as it streams, so the feedback control has a real id by the time the reply finishes. Replies restored from history on a page reload carry their id already, so older turns can be rated too.

What is stored

One row per rated message in message_feedback, scoped to the agent (and therefore the workspace) and holding:

• the rating, the reason category, and the visitor's comment;
• snapshots of both the question and the answer — the record has to stay readable after the conversation is pruned, since "this exact answer was wrong" is worthless once the text is gone;
• the page the visitor was on, and the conversation id for the deep link.

message_id is unique, so a visitor who changes their mind updates their existing row instead of stacking duplicates.

Reading the page

• Headline stats — total ratings, helpful, not helpful, and the negative rate, with the change against the previous window of the same length. These always follow the date window and ignore the rating / reason pills, so the rate stays meaningful while you filter.
• Why visitors were unhappy — the reason breakdown, and the fastest diagnosis on the page. A rising Not correct means the knowledge base holds something wrong; a rising Incomplete usually means a page was never crawled. Click a category to filter to it.
• The list — question, answer, reason, comment and page for each rating, newest first, capped at 100 rows. Defaults to negatives only, because those are the ones that need work.

Closing the loop

Each negative row offers Write a curated answer, which opens the agent's curated-answer form with the failed question pre-filled. Save it and the next visitor who asks that question gets the pinned answer instead of the LLM — the same path Content gaps uses to turn a complaint into a fix.

There is deliberately no email per bad rating: gap notifications already cover the "something needs attention" signal, and one alert per thumbs-down would be noise. The page is the weekly read.