Content gaps (self-improvement)

Pitchbar's self-improvement loop watches every conversation and flags questions your agent could not answer confidently. Those questions cluster into content gaps at /app/analytics/content-gaps. Treat the page as a prioritised backlog of what to teach your bot next.

How a gap is detected

After every visitor turn — once the stream has already emitted done, so the visitor's latency is unaffected — DetectGapJob runs inline (dispatchSync) with the user message, the agent id, and the conversation id. It is deliberately NOT left on the analytics queue: gap-recording is the only visible surface of the self-improvement loop, and a production worker started without that queue in its list would silently strand every gap unwritten (blengi, 2026-07-06). Running it inline — the same guarantee PersistTurnJob has — means a misconfigured worker can never drop a gap. The follow-up email is the only queued part (see below). The job:

1. Normalises the question (lowercase + collapse whitespace) and hashes it.
2. Snapshots up to the last two turns before the gap question (role + trimmed text) from the conversation, so the recorded question can be understood in context.
3. Looks up an existing gap with the same hash for this agent; if found, increments occurrences, refreshes last_seen_at, and re-points the conversation_id + context at this latest occurrence (a null conversation from a legacy caller never overwrites a good snapshot).
4. Otherwise inserts a new row in content_gaps with status=open, storing the conversation_id + context snapshot.

The job is single-attempt — duplicating a gap row is harmless, but retrying after a transient DB hiccup would over-inflate occurrences.

What counts as a deferral

A turn feeds a gap when retrieval came back low-confidence or the reply itself reads as "I can't answer that". The reply check is deliberately fuzzy and multilingual: it recognises the strict-mode fallback even when the model drifts from the template ("cannot" for "can't", an apology prefix, a reworded token) or answers in the visitor's language instead of the agent's — a Dutch agent deferring to an English visitor still logs the gap. Common refusal phrasings in English and Dutch ("I couldn't find…", "dat weet ik niet…") are caught even with Strict Mode off. Confident answers that merely reuse the same vocabulary ("Based on the available information, the price is…") are not flagged.

What the UI shows

• Agent picker — narrow to one bot when you have several.
• Status — Open (default), Answered (someone added a curated answer / source), Ignored (won't fix).
• Window — last 7 / 30 / 90 days, or all time.

Each row carries the question text, the count of times it has been asked, the last time it was asked, and the agent it belongs to.

Because a captured question often leans on earlier turns — a visitor who asks “Where does Jan Roel live?” and then “Is he connected to Alles Opslaan Meppel?” leaves the second question meaningless on its own — every row also shows a small Conversation context block with the couple of messages that came just before the gap (labelled Visitor / Agent), plus a View conversation link to the full transcript. The context is a snapshot taken when the gap is recorded, so it survives even if the original conversation is later pruned; the link points at the most recent conversation the question was seen in.

Actions

• Create knowledge — jumps to the curated-answers editor for that agent with both the trigger pattern and the knowledge-base title pre-filled from the visitor's original question. When you save, the answer is created and the gap is automatically marked answered — you don't have to come back and resolve it by hand. The next visitor who asks that question gets a direct response without going through retrieval.
• Resolve — manually marks the gap as answered (e.g. you added a knowledge source that now covers it).
• Ignore — marks ignored (out of scope for this bot, e.g. visitor asked about a product the workspace doesn't sell).

The loop also closes from the suggested-answer path: when you Approve a system-suggested curated answer that was drafted from a gap, that gap is marked answered in the same click.

Email notifications

You don't have to keep the gaps page open to catch new gaps. Each owner/admin picks a cadence under Email me unanswered questions on the /settings/profile page:

• Off — never emailed.
• Immediately (the default) — the first time a distinct question is detected, DetectGapJob emails you straight away. One email per gap: a recurring question increments occurrences but never re-emails, stamped by content_gaps.notified_at.
• Daily digest — once a day, a single email of every gap seen in the last 24 hours, most-asked first.
• Weekly digest — the same, batched over the last 7 days. Both digests are sent by the pitchbar:gap-digest scheduled command and skip you entirely on days with no new gaps.

Whichever cadence you choose, every email — immediate or digest — carries the visitor's question, the agent, the workspace, when it was asked, and a one-click Create knowledge button that lands you on the curated-answer editor with the question prefilled and the gap wired for auto-resolve.

• Per owner. The cadence is per person — one owner can take immediate alerts while another takes a weekly digest. Editors and viewers never receive these.
• Localised. Every email renders in each recipient's own language (their profile locale), so a Dutch owner gets a Dutch email.

Who can do what

Why this matters

Most chatbots silently degrade — a question never gets answered, the visitor bounces, no one notices. Pitchbar turns those silent failures into a triage queue. Working the gap queue weekly is the fastest path to a high-quality agent.