Repository navigation
A failed embedding is reported to the agent as an empty knowledge base — the one failure RAG cannot afford to state as a fact #628
Description
search_knowledge answers a broken embedder with the sentence "the knowledge base may be empty". That sentence is a fabricated explanation of an infrastructure failure, and it is the one the model reads.
The path (all verified)
embed() swallows everything:
// workers/api/src/agent-storage/vectors.ts:304
protected async embed(text: string): Promise<number[] | null> {
if (!this.ai) return null;
try { ... } catch { return null; } // :320-322
}grep -in "log\|error\|console" workers/api/src/agent-storage/vectors.ts returns 0 lines inside embed — a Workers-AI outage, a rate-limit, a 500 from the model gateway all become null with nothing written to the error log, the trace, or the console.
vectorSearch turns that null into "no matches":
// vectors.ts:106-109
if (!this.vectorize || !this.ai) return [];
const embedding = await this.embed(query);
if (!embedding) return [];Two callers consume that empty array, and both state it as a fact about the user's data:
-
The tool.
workers/api/src/lib/storage-tools.ts:262-267—
if (results.length === 0) return ok(call.name, "No relevant results found. The knowledge base may be empty or the query didn't match any stored content.")
Noteok(...), notfail(...). The model is told the search succeeded and found nothing. -
The prompt.
workers/api/src/agent-storage/context.ts:71-72—
const results = await this.vectorSearch(query, 8); if (results.length > 0) { ... }
On an empty result the entire## Relevant Knowledgeblock is omitted. The turn proceeds ungrounded, and nothing in the prompt, the trace orlist_errorssays retrieval ran at all.
Why this is the read path's version of a bug the write path already fixed
indexingEnabled exists precisely so this cannot happen on ingest:
// vectors.ts:26-31
* ... so callers MUST check this before reporting a KB add / repo index as "indexed" —
* otherwise ingest looks green while RAG is dead (#22).
get indexingEnabled(): boolean { return !!(this.vectorize && this.ai); }grep -rn "indexingEnabled" workers/api/src — the read path never consults it. #22 stopped ingest lying about indexing; the symmetric lie on retrieval was never closed.
Blast radius (live, read over MCP)
Repo Chat(267286d5-6877-4f2a-8b5c-3c40b3e3be85) —vector_stats: 315 sources, 3,979 chunks, 1,823,670 chars. Its declared tools are exactly["search_knowledge","list_knowledge","read_knowledge"], sosearch_knowledgeis its only route to any of it.- Every RAG agent shares
buildRAGContext: Doc Chat, Local Repo Chat, Google Drive Doc Chat, the apply agent's KB.
An agent told "the knowledge base may be empty" will report that to the user, and the user will believe their ingest failed.
What "fixed" looks like
Distinguish the search returned nothing from the search did not run. embed() should report its failure (a distinguishable return, plus one logError so it is visible in list_errors), vectorSearch should propagate it rather than flattening to [], search_knowledge should fail(...) with "retrieval is unavailable right now" instead of ok(...) with a claim about the corpus, and buildRAGContext should say retrieval was attempted and failed rather than silently omitting the block.
Verified vs inferred
- Verified: every line and grep above; the
Repo Chatnumbers are a livevector_statsread on 2026-08-16. - Inferred: the frequency of embed failures in production. Not measurable today — that is the point of the issue: there is no record of one ever having happened.
Activity
fix(rag): a search that could not run stops being reported as an empt…
serge-ivo commentedon Aug 15, 2026
Fixed in 50192bc (on
main, CI green, API Worker deployed).What each surface says now
search_knowledge→ok: "No relevant results found. The knowledge base may be empty…"search_knowledge→fail: "Knowledge retrieval is unavailable right now — the search could NOT be run, so this is not a statement about what is stored. Do not tell the user their knowledge base is empty…"## Relevant Knowledgeblock silently omitted## Knowledge Retrieval Unavailableblock: "…its absence is not evidence that it does not exist"ok: "…may be empty or the query didn't match"ok: "No relevant results found. The search ran successfully and matched nothing — the knowledge base may be empty, or the query didn't match any stored content."logErrorper failure, sourcerag-retrieval→ readable inlist_errorsand mirrored into the traceThe empty-corpus case deliberately stays a plain
ok. Collapsing both into a generic error would have traded one lie for another, and an empty corpus is a legitimate, common, correct result.How it was done
Mirrored
indexingEnabledrather than inventing a second vocabulary, as the issue asked.embed()returnsnullonly when there is no AI binding — precisely whatindexingEnabledalready reports; every other outcome throwsRetrievalUnavailableError. A 200 carrying no vector counts as a failure too: with AI configured, that is a provider failure wearing a success's clothes, which is the same callvectorizeStorehas always made.vectorSearchpropagates it, and now wraps a Vectorize outage the same way. That closes an asymmetry the issue did not name: an embedder outage was silent, while a Vectorize outage propagated raw out ofbuildRAGContextand 500'd the whole chat turn. Both are now the same honest answer.New vocabulary lives in
workers/api/src/lib/retrieval.tsso the sentence and the failure report each have one definition.Deliberate boundary — please sanity-check this call
indexingEnabled === false(no Vectorize/AI binding, e.g.PLATFORM_AI_ENABLEDoff) still returns[], not an error. Reasoning: that is a static property of a deployment on which nothing was ever embedded, so there is genuinely nothing to find — it is not an event that makes a populated corpus look empty, which is the failure this issue is about. #22 already requires the write side to reportvectorized: falsethere. Raising it on every turn of an AI-disabled deployment would be noise. Say the word if you want that case to speak up too.Rejected
Having
embed()log its own failure (the issue suggests "plus onelogError"). The storage engine has noEnv, and itsmeter— the existing{DB}escape hatch — isnullon the storage-route path becausewithEngineresolves the engine without a userId. So the report is raised by the two callers that do have anEnv, which is the shape #22 already uses. Thelist_errorsvisibility the issue asked for is delivered; only its location moved.Also rejected: leaving the usage ledger inside the embed catch. It was folded in, so a D1 hiccup in
recordPlatformUsagediscarded a perfectly good vector. It now fails independently of the embedding it is accounting for.Verified
19 tests in
workers/api/src/agent-storage-retrieval.test.ts, each watched failing first against the restored old behaviour — 7 did, includingexpected '' to match /Knowledge Retrieval Unavailable/for the silent prompt block. Both directions are pinned: a fix that answered every empty result with "retrieval failed" fails this suite too.Full bar green (typecheck ×11, worker test typecheck, biome, and all 9 script gates) at the commit, checked in a clean worktree to exclude other lanes' concurrent edits. Playwright e2e skipped — server-side API Worker only, no
store/oragents/coder/web.Not verified
That an embed failure now renders correctly in production: I cannot force a Workers-AI outage without causing one. The success path is verified live instead. The frequency question the issue flags as unmeasurable is now measurable —
rag-retrievalinlist_errorsis the counter that never existed.serge-ivo commentedon Aug 15, 2026
Production check after the deploy — partial, and I want the limit on record rather than buried.
Deploy of
947fe832to the API Worker completed 22:47:51 UTC; read taken 22:55:54 UTC.No regression signal:
vector_statson Repo Chat (267286d5-…) is byte-for-byte unchanged: 315 sources / 3,979 chunks / 1,823,670 chars,lastIndexedstill 2026-08-04. The store was not touched by the deploy.list_errors(124 rows, 2026-07-23 → 2026-08-15): occurrences ofrag-retrieval= 0. No new error source in the window, and no error at all in the last hour.@cf/baai/bge-base-en-v1.5shows 243 calls / 8,106 input tokens over 7d with no associated errors — the embedder itself is healthy.What that zero does and does not mean. It rules out a deploy-time or startup burst of retrieval errors. It does not confirm the success path, because nothing exercised it: the latest
lastActivityAtacross all 40 instances is 2026-08-15 21:43 UTC — over an hour before the deploy — and Repo Chat's most recentsearch_knowledgecall was 2026-07-07. Zero errors is currently consistent both with "the fix is fine" and with "no search has run", and the activity data says the second is why.So: closed on the strength of the unit tests (19, each watched failing against the restored old behaviour) plus the absence of any regression signal — but the first real search on a live agent is what actually confirms the success path. If anyone runs one, re-checking
list_errorsforsource: "rag-retrieval"turns this into a genuine pass/fail. I'd rather say that than claim a green I did not measure.One incidental finding, unrelated to this issue and worth its own ticket if it recurs:
list_errorshas nosourcefilter, and at 124 rows its response is ~69 KB — enough to overflow a 25k-token MCP host budget in a single call.serge-ivo commentedon Aug 15, 2026
Success path verified live — the check
50192bcacould not makeThe fix was deployed with the honest caveat that nothing had exercised the search path since (the newest activity across all 40 instances predated the deploy by an hour), so a zero
rag-retrievalerror count ruled out a deploy-time burst but did not confirm retrieval still worked.Run now against
267286d5-6877-4f2a-8b5c-3c40b3e3be85(Repo Chat — 315 sources, 3,979 chunks, reachable only through this path):Both results are semantically on-target, so
embed()→vectorSearch→ results is intact end to end. The fix narrowedembed()'snullreturn to the no-AI-binding case and made everything else throw; this confirms the throw did not swallow the ordinary path.Still not exercised, and worth stating rather than implying otherwise:
So: one of three surfaces confirmed live, two pinned by test. The one confirmed is the one that would have been most costly to break, since it is the path every Repo Chat question takes.
fix(security): the knowledge tools return the corpus fenced, the same…