Repository navigation
feat(rag): LanceDB retrieval, cited answers, and PDF fidelity fixes - #3
Merged
Merged
Conversation
|
Too many files changed for review (135 files, 100 file limit). Bypass the limit by tagging |
Adds a retrieval layer and an Ask/RAG Lab UI over parsed documents, and fixes several pipeline defects found while running two real ICAI study PDFs through it. Retrieval (wikify/rag/) - LanceDB store with local model2vec embeddings (256d, no API key, no server). - Four modes: vector, full-text, hybrid (RRF), and filter. Filter returns EVERY match — "give me all X" is a metadata question, not a similarity guess. - Contextual-retrieval prefix on embed text, parent-section expansion, optional LLM rerank that degrades to fusion order when unavailable. - Permissions are a LanceDB pre-filter, never a post-hoc trim. An omitted ACL decision throws rather than searching everything. Answering - Intent router (exhaustive/semantic/hybrid) with follow-up rewriting; the decision and its plain-language reason are shown in the UI. - Answers cite [n] markers; each citation carries a resolved page, line span and verbatim quote, verified against the source before display. Verification is two gates: fuzzy prose match AND exact equality on figures, signs and statutory refs — a flipped (+)/(-) previously passed a 0.85 similarity check. - Honest refusal when nothing retrieved clears the bar. - Per-question cost and token usage returned and displayed. - Conversations persist to Wikify Ask Session/Message, so route decisions and citations become an evaluation set. Pipeline fixes - Sectioning stopped at the first heading longer than the 140-char title column, after the old tree was deleted — silently dropping 93.6% of a 236-page document. Titles are now clipped; coverage went 6.4% -> 99.8%. - Page classification gated on `chars < 250 AND drawings > 40`, which no born-digital diagram page can satisfy; replaced with per-region detection (1/236 -> 231/236 pages routed correctly). - Tables were being encoded as mermaid flowcharts, destroying row-to-rate correspondence; they now emit HTML tables, and diagrams are parsed and repaired (quoting node labels, `&` chains) before storage. - Page verdicts were frozen at the baseline parse, so remediated pages still read "review"; they now track the adopted content, with a backfill patch. - Remediation could adopt a near-empty candidate over a good one. - Page edits now propagate into the sections and index built from them. Frontend - /ask and /rag-lab, with sources rendered before the answer, citation chips, provenance, and a naive-vs-routed comparison that shows what similarity search missed. Unranked results no longer render a meaningless full score bar. - Mobile support across the app: shell navigation, drill-down replacements for desktop splits, and graph views that degrade to a list rather than an unreadable canvas. Evaluation - 12 golden questions with recall, precision and completeness, plus an HTML scorecard. Routed retrieval beats naive 66% -> 87-90% recall on the demo corpus; routing is an LLM call, so treat the figure as a range.
Rl0007
force-pushed
the
feat/rag-chat-lancedb
branch
from
August 11, 2026 03:20
53ac237 to
59e6ce2
Compare
Twelve images covering the Ask flow with citations, page/line provenance, the cost meter, the naive-vs-routed comparison, the refusal state, a rendered mermaid diagram, the reconstructed ICAI surcharge tables, the page verdict list, and three mobile views plus dark mode.
Three defects found by testing the Ask path against two real ICAI documents, plus the cleanup pass that followed. Reranking — answers were being refused that the corpus covered - `search()` re-sorted by fusion score after reranking and then applied the limit, so the reranked winner was pushed back to its fusion position and sliced off. "Slab rates under section 115BAC(1A)" scored 8.0 at fusion rank 10 and never reached the answer; the question was refused. One `rank_key` now drives both the rerank and the slice. - Grading 35 candidates in one call makes the classifier emit a run of zeros. Batching at 10 returns real scores. A batch whose reply skips candidates is dropped whole rather than letting the missing ones read as a confident 0. - Refusal now needs two legs to agree: a below-floor rerank is overruled by a strong embedding match. Calibrated on both corpora — answerable questions peak 0.56-0.63, unanswerable 0.39-0.45. Genuine refusals still refuse. - When the embedding leg overrules, citations carry `rerank_score: null` rather than 0.0, so the UI stops showing a dead relevance chip. Cost — the reported figure was ~15x too low Only the price printed on the response body was counted. The router and reranker use the REST client, which returns cost inline; synthesis goes through litellm, which reports price to a callback on its own thread. So the meter added $0.00 for the leg carrying ~92% of the spend. A `CustomLogger` keyed on `litellm_call_id` now bills each call back to the collector that started it, and the second accounting mechanism in `history.py` is deleted — session totals are summed from message rows so the header cannot drift. Verified against OpenRouter's own generation records: an ICAI question costs $0.047, not the $0.003 previously reported. Efficiency - Rerank batches run concurrently. `frappe.local` is unbound on a pool thread, so the API key is resolved on the calling thread and passed in. Measured 19.81s -> 16.00s median per ask, with identical call count and spend. - `index_status` did one LanceDB connect and one full scan per readable project; now one scan with a single `project IN (...)`. - Page propagation rebuilt the whole FTS index once per section and re-read every page of the document each time; now one batched upsert and one FTS rebuild, loading only the relevant pages. - Sectionising fired the reindex hook per row (~592 redis round trips, 295 of them no-ops); it now suspends per-row indexing and queues one rebuild. - `get_page_image()` ran per page inside the remediation loop; the field is read with the rest of the row. - Mermaid sources were parsed up to nine times per page; now once. - `find_regions` read each page's drawings twice. Reuse and dead code Duplicate `get_pages_by_document` / batched-IN / page-line-span helpers collapsed to one definition each; `useIsMobile` folded into `useMediaQuery` so the app has one media-query registry; AskWiki reuses `MarkdownPreview`, which also gives Ask answers mermaid rendering they never had; the eval harness now calls production's `retrieve()` and `compare()` instead of copying them, which is the drift it exists to catch. Removed an unreachable `EvalScoreboard` branch, a dead `fts` relevance basis, an unused `page_regions` parameter, and a `wait_for_prices` call that could add 5s to a failing ask without affecting any reported number. Streaming answers no longer re-parse the whole markdown per token. References to five spec files that are not in this repo are rewritten to point at the code that holds the information.
Asked "how many job descriptions are in the Demo Corpus project?", the agent
answered "there are none". There are 15. Two faults produced the same sentence,
so a correct chain of reasoning ended in a false statement.
- The `query` argument was a raw substring filter over title and hierarchy
path. Questions are plural ("job descriptions"), titles are singular
("Job Description — Theatre Scrub Practitioner"), so every match was
filtered away. It is now a narrowing hint: token-wise, normalised for case,
punctuation and trailing plurals, and when it matches nothing the full set
is returned with the filter reported as ignored rather than as absence.
- The model passed the project title, not its id. `Ctx.default_document`
already guarded against exactly this, with a comment noting models echo
display labels instead of bare ids; the project field had no such guard.
Added `Ctx.default_project`, and closed the same hole in `semantic_search`.
Three outcomes that were indistinguishable now read differently: an unknown
section type (with the near matches, and an explicit note that this is not
evidence the content is missing), a valid type that is empty in scope (naming
the scope and which types do have content), and a filter that removed
everything (naming how many it removed). Every reply carries the in-scope
count and the resolved scope, so absence cannot be inferred from silence.
Also closes a fixture leak: types created by tests that drive the agent loop
survived rollback because the loop commits mid-turn, and were being offered to
users as real taxonomy suggestions.
OpenRouter routes by price by default and re-draws a provider per call, so concurrent rerank batches were scattering across endpoints of different speed and the wall clock was set by the slowest draw. Pinning the endpoint removes the draw. Measured over 5 batches, 9 runs each: unpinned 7.8s median with 1.9x effective concurrency and a 24.7s worst case; pinned 2.6s median, 4.2x, 4.1s worst case. The latency was the smaller half. Six of eighteen unpinned runs returned truncated JSON, which discards the whole rerank verdict and drops the answer back to fusion order — against zero of eighteen pinned. That is the most likely source of the intermittent all-zero rerank we chased earlier, and it means provider scatter was corrupting results, not just slowing them. `allow_fallbacks` stays on: a reranker that cannot reach its preferred provider must degrade to a slower one, never fail. Retrieval with reranking measures 8.9s to 5.8s end to end. Total ask time is unchanged in a single sample because the router leg drew slowly on that run; the router change here is the same one-line pin, so that reading needs repeating before anything is concluded from it. Also raises the rerank worker cap now that batches no longer contend for a provider draw.
test_ask_cost pins the per-ask figure against the sum of its individual LLM calls, and the session total against the sum of its message rows. Without it the 15x undercount could return silently: litellm reports synthesis price via a callback rather than on the response, so the leg carrying ~92% of the spend read as $0.00. docs/rag-latency-prior-art.md records what production systems do about the same problems, with sources and dates. Its main finding contradicts the assumption we started from: synthesis is decode-bound, so trimming the 55k character prompt is a cost fix rather than a speed fix, and the reranker's model is the bottleneck rather than its depth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
`session` named two different things. The interface minted a per-ask correlation token so it could pick its own deltas off the per-user realtime channel, and sent it as the `Wikify Ask Session` docname the backend records turns against. Two consequences. Every ask opened a new conversation, so a follow-up was never rewritten against what came before it. And `recent_turns` ran `has_permission` on a name that had never existed, which loads the document and raises DoesNotExistError — the first non-Administrator to ask a question got a 404. Administrator short-circuits the permission check, which is why this survived testing. The token is now `stream` and is only echoed back on realtime payloads; `session` is the docname, absent on the first ask and replayed from the response after that. Switching project starts a new conversation rather than rewriting a follow-up against turns about other documents. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
Page propagation hung off `Source Page.on_update`, so in production it almost never ran: every real write to `canonical_markdown` goes through `store.set_canonical` or `store.set_canonical_markdown`, and both use `frappe.db.set_value`, which fires no doc_event. A re-remediated page was correct in Page Review while the sections built from it — and every chunk indexed from those sections — kept serving the old text. Only the tests exercised the hook, because they saved the document through the ORM. Invalidation now hangs off `engine.store`, which already calls itself the write funnel, and the tests drive it the same way. `events.suspended_indexing()` covers the bulk passes in remediate and finalize, which write every page and then replace the tree wholesale — invalidating page by page inside them would queue a redundant pass over work that is about to be thrown away. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
The rebuild deleted a document's section tree and then inserted the new one with nothing tying the two together. An insert that threw left the document holding whatever prefix of the new tree had landed and none of the old — and jobs.remediate's handler reverts the import to Review on the stated premise that the parse result is intact, then commits, so the truncated tree became the document. Delete and inserts now share a savepoint that is rolled back on failure, leaving the previous tree standing. The over-length title that first exposed this is clipped, but the hazard was the missing savepoint, not that one trigger. The project rebuild also moves out of the finally block: on failure the old tree is what the index already holds, so restoring it needs no rebuild. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
The gate re-implemented enough of mermaid's grammar in regex to decide whether a block would parse, while the reader renders it with the real parser client-side. Every valid construct the regex did not know was read as broken and the diagram discarded — `A & B --> C` is valid mermaid 11 and cost page 6 its diagram twice before someone special- cased it, which is the shape of the whole problem: the approximation is only ever behind the grammar it is chasing. Syntax now fails where the real parser lives; utils/mermaid.js already catches the parse error and renders an error chip over the block's own source. What stays is the judgment regex is actually good at and the reason this module exists: reading nodes and edges to recognise a rate table flattened into two unbound columns, where a confidently wrong rate is worse than no diagram. Every block is now passed through the lossless quote repair unconditionally, since deciding whether it was needed would mean judging syntax again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
… place `set_canonical_markdown` replaces a page's text and scores nothing, but left the old verdict and canonical composite on the row — so an agent edit could leave "pass 0.99" sitting on content nobody had ever verified. The canonical-verdict fix only covered `set_canonical`, the seam that carries a new composite, so this shape survived it. Cleared rather than re-derived: scoring needs the page image and the baseline, which is a verify pass, not something a write seam can do. Unscored reads as verdict "" and composite 0 — the value `backfill_canonical_verdict` already skips, and the review UI already renders an empty verdict as a gray dash and counts it as flagged. The live caller is the agent's `edit_page_content`. `finalize_document` also writes through this seam but has no production caller today. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
G1's naive leg was pinned at 5 correct from before the corpus was re-sectioned; it measures 6 now, deterministically, across repeated runs. The PR's own headline already quotes 6 correct and 9 missed, so the constant was the thing lagging, not the number. Naive got better here, which narrows the gap the POC argues rather than breaking it: routed still returns all 15 across all 5 documents. The two phrasings now agree at 6, but stay asserted separately so a future divergence is visible rather than averaged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
The figure predates the coverage, sectioning and reranker fixes, and both failure modes it turned on have been worked on since — so it is not a claim about the system as it stands. Marked as stale in both directions: whether the grade moved is unmeasured. Also records why it cannot simply be re-run: the twelve questions were asked by hand and only the score survived, so there is nothing to reproduce or compare against. The next grading should transcribe its question set the way rag/eval.py does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
Resolves NewImportDialog.vue in favour of main's batch PDF upload: this branch only carried the older single-file FileUploader block, which that feature replaces, and the rest of the file had already auto-merged to the batch design. The resolved file is identical to main's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
Replaces the batched OpenRouter reranker with an in-process ONNX cross-encoder. Scoring is total and deterministic, so the batching, the concurrency pool and the provider pinning that existed only to stop the LLM returning short or all-zero verdicts all go away. A rerank now costs no tokens and works on a site with no LLM configured at all, where an absent key used to skip it silently. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
An exact-match cache keyed on the question, scope, history, model and the store's write counter, so it can never outlive the corpus it answered from. A hit replays the same realtime sequence the live path publishes, with zero spend and `cached: True`; refusals are never stored, since they can come from a transient failure rather than a fact about the corpus. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
The lab was a development surface; /ask is the product. With it gone the mobile tab bar has room again, so the overflow sheet is back to user actions only and the assistant returns to a floating button on desktop. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
frappe-ui pins the panel to sm:min-h-[560px], which beats the height cap below a 688px viewport: the panel runs off-screen and clips its own header and Save button with nothing to scroll them back. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
`@frappe.whitelist()` with no `methods` allows GET, and Frappe runs `validate_csrf_token` only on non-GET requests. So every state-changing and billable endpoint was reachable from an `<img src=...>` on any page a logged-in user visits: `rag.ask` and `rag.compare` spend real OpenRouter money and write session rows as them, `rag.reindex` and the import triggers enqueue `timeout=3600` long-queue jobs in a loop, and the whole of `sections.*` edits the tree. 26 endpoints across rag, imports, agent, projects and sections. The 16 that stay GET are reads. `ask_history.delete_session` already had this and is the pattern. No frontend change: frappe-ui's `call()` is unconditionally POST and sends the CSRF header, and every `useCall` site for these methods already declared `method: "POST"`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
`start_import`/`start_imports` wrote `file_url` verbatim to `Wikify Import.pdf` and enqueued a worker to parse whatever it pointed at. Any user could name another user's private attachment and have its contents re-published as wiki sections; the batch endpoint multiplied that by MAX_BATCH per call. Checked per File row rather than through `get_list`, because File's read rule lives in a `has_permission` hook that a query filter never runs. The batch checks every url before creating any Import, so one unreadable file enqueues nothing rather than half a batch. Fixtures now create real File rows (a one-page PDF via fitz, the house pattern from test_parse_pipeline) since a url with no File behind it is no longer importable. Verified red-green: the two batch guards fail with the check removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
`Wikify Ask Message` granted role All create + write + delete with `if_owner`, and `if_owner` does not constrain `create`. Any logged-in user could insert a row pointing `session` at a conversation they do not own, poisoning its transcript and the cost totals `set_totals` sums from it. Role All is now read-only `if_owner`. Turns are written by `history.append_message` with `ignore_permissions`, which is justified: the conversation is the object of authority and `record_turn` checks write on it first. The `on_trash` cascade uses `frappe.db.delete`, so dropping delete costs it nothing — pinned by the existing test_deleting_a_conversation_removes_its_turns. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
`remove_unverified_diagrams` returned "" for any block that failed the meaning gate, destroying the model's structured read of the page with no way back. The gate has a known false-positive class: MIN_VALUE_LEAVES = 3 trips on any fan of three bare values, which on a rate referencer is most flowcharts. The block is now demoted to a plain `text` fence with a one-line note, so the reader sees what was meant and that it did not draw — which is what this module already does for a diagram the renderer rejects on syntax. The source-crop fallback is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
0.7 moved page invalidation onto the write funnel because `frappe.db.set_value` fires no doc_event. The link below it was left open: `set_section_markdown` calls itself THE write funnel for `Source Section.markdown` and writes the same way, so the `Source Section` reindex hook never saw it either. An agent or UI edit changed exactly the text the index serves and the index kept serving the old copy — the bug this branch is named after, one link further down. Two seams, both batched: - `section_content_changed` reindexes the sections a write touched. Called from `set_section_markdown` and from `set_section_type`, since the type is a filter column every chunk carries. - `document_structure_changed` queues one project rebuild from `_rebuild_tree`. `hierarchy_path` is embedded into each chunk as its contextual prefix, so a rename or a reparent invalidates the whole document rather than one row; the seven tree APIs that rebuild all route through there. `toggle_include` is deliberately not wired: `include_in_wiki` is not a chunk field. Verified red-green: the markdown-edit test fails with the funnel call removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
The branch pushes `readable_projects()` into the LanceDB `where` clause and makes an omitted ACL decision throw. Two paths went around it. `index_stats(projects=None)` defaulted to reading every project on the site — the exact permissive default the AclDecision sentinels were introduced to make impossible (see the note above ALL_PROJECTS). It now takes ACL_REQUIRED and calls the same `assert_acl_decision` guard; `ALL_PROJECTS` is how a background caller opts out. `Ctx.default_project` resolved any Wikify Project by title with no permission check, and the resolved id flows into the agent's read tools and `api/explore.py`, which had no permission checks at all. So a prompt could widen the agent from its attached project to any project on the site. Resolution now goes through `get_list`, and Explore carries the same ACL as retrieval. Explore subtracts unreadable projects rather than scoping to readable ones. A document outside every project carries no permission statement and is visible today; an allow-list would have hidden it as a side effect of a permission fix. `readable_projects` / `assert_readable` move to `api/permission.py` so retrieval and Explore cannot drift, and that module's docstring stops claiming the Wikify DocTypes are System Manager-only — this PR ships two that are not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
…stion Both retrieval models were pulled from the HF hub on first use, inside `search()`, which `ask()` reaches on the web worker with `rerank=True` by default: the first question after a deploy paid ~90MB of ONNX plus the embedder, once per worker process, and an air-gapped bench hung on an HTTP timeout inside the answer path. Neither download named a revision, so the hub could serve different weights tomorrow — and the 0-10 rerank map, `answer.MIN_RERANK_SCORE` and every calibrated score floor are measured against the logits these specific weights produce. Both are now pinned to the commits the eval harness was measured on. model2vec's `from_pretrained` takes no `revision` and defaults to `force_download=True`, so the embedder resolves a `snapshot_download` and loads from the local path — which also stops it re-fetching on every cold start. `rag.warm.warm_models` runs on `after_migrate` and stands alone as a bench command. It never fails the caller: a host with no network gets a logged error and the failure surfaces at first use instead of breaking a migrate. A missing embedder now throws with the command that fixes it rather than an HF traceback — every mode except `filter` embeds the query, so there is nothing to degrade to. Verified: warm_models returns both models loaded, and the 17 golden-question eval tests pass against the pinned weights. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
…hooks Four mechanical corrections from review. `search()` took a bool named `rerank` while the module imports the `rerank` module — the parameter shadowed it, and only the fact that `rerank_hits` resolves the module from global scope kept it working. The next `rerank.scores(...)` written inside `search()` would have raised AttributeError on a bool. Renamed to `use_reranker`; the whitelisted `rerank` argument stays, since it is the wire contract the SPA sends. `api.rag.search` clamps `limit` to MAX_LIMIT. It widens the candidate list, and with the reranker on each candidate is a 512-token cross-encoder forward pass on the web worker inside the request — unbounded, an authenticated way to spend the whole box on one call. `replace_sections` uses `events.suspended_indexing()` rather than a second hand-rolled copy of it, and the savepoint takes a random name the way Frappe's own helper does, since re-declaring a name destroys the outer savepoint holding it. The no-commit invariant the rollback depends on is now written down. `hooks.py` drops 229 lines of `bench new-app` commented-out template; the live statements are byte-identical. Frappe dependency narrowed to `<17.0.0` — v17 does not exist and cannot have been tested against. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
The key carried the raw question plus the whole conversation history, so every turn after the first was unique to its session and only a session's opening question could ever hit — the opposite of the shared-syllabus traffic the module's docstring justifies the cache with. `ask` now routes before it builds the key and hands the decision to `answer()`, so the key is the question the router RESOLVED: two readers whose follow-ups mean the same thing share an entry, and two whose follow-ups differ still cannot collide. `usage.collect()` nests, so moving the routing call out of `answer()` keeps every leg of the turn in one total — and a cache hit now reports the routing call it actually made instead of claiming to have cost nothing. Two more holes in the same key: - CACHE_VERSION. `index_version` retires an entry when the corpus moves, but nothing retired one when the synthesis prompt or a score floor moved, so a deploy kept serving answers written by the old behaviour for a full 24h TTL. - A store that could not be opened returned version 0 — the same namespace a genuinely unindexed site uses — so an entry written during a blip outlived the corpus it answered from. It now yields no key at all, and the turn skips the cache. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
Prose was 30% of these files (1,132 of 3,801 lines). It is now 6%: every docstring and every comment goes, except the ones that record a measurement, a workaround or a gotcha — those survive as one-liners next to the code they explain. Kept, because they are the reasons the code is shaped this way and cannot be re-derived by reading it: the 512/320/256/128-token rerank recall figures, the observed logit range the 0-10 map is built on, the RRF rank-based fusion scores (0.0328 for both a perfect and a nonsense hit) and the vector floors calibrated against them, the context-window trial that cut spend 55% and citations 12 -> 7, the figure-comparison finding where "(+)" retyped as "(-)" scored 0.85 against a 0.82 floor, the `cache.get_value` stale-read that made coalescing swallow changes, the doc_event that never fired, the savepoint's no-commit invariant, and every `# ponytail:` marker. Verified mechanically: for all twelve files the AST with docstrings removed is identical before and after, so nothing but prose changed. One test fix rides along, because the strip exposed it: the ask cost test asserted against a live answer while the answer cache could serve it a previous run's entry — it now bypasses the cache explicitly. It was latent before this change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
Re-picking the project on every visit was the most repeated action on the page. The selection is now held in localStorage — per-browser and per-viewer by nature, so it never travels between users, and a browser with site data blocked simply doesn't remember it. A remembered project the viewer can no longer see (deleted, or permissions changed) is dropped as soon as the options load: `ask` refuses an unreadable scope outright, so a stale selection would fail every question until they noticed the select. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
An empty thread now opens on a centred composer under a time-aware greeting rather than a dashed empty box; the first turn moves the composer to the bottom, where a follow-up belongs. Sample questions sit under it and retire with the landing. The conversation id rides in the path (/ask/:conversationId), so a refresh, a bookmark or a pasted link reopens the thread instead of dropping it. The transcript only follows a streaming answer while the reader is already at the bottom of it — scrolling up to re-read no longer gets yanked back down on every delta. The composer is extracted to AskComposer so its two placements cannot drift apart, and a citation carries its import so a source card leads somewhere before the wiki is generated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
Opening /ask/:conversationId painted the landing — greeting, composer, suggestions — for the length of the fetch before the thread replaced it. That flash reads as "new chat", the opposite of what the link said. The restore now holds a turn-shaped skeleton instead. The composer takes its placement as a prop rather than inferring it from the turn count, which got the same case wrong: a restored thread's turns arrive after the fetch, so the composer wore its landing face under a thread that was already there. Sample questions are now ones measured against the live index. Retrieval applies a relevance floor and the previous set sat under it — "Summarise the key sections" scored 2.8 and "What is still open or unresolved?" 1.7 against a floor of 3.0, so both answered "not in this wiki", which is worse than offering nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
Rl0007
force-pushed
the
feat/rag-chat-lancedb
branch
from
September 9, 2026 06:35
6cee175 to
2745c8a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implementing RAG based Search, you can select a project and converse on it.
Screen.Recording.2026-09-09.at.3.29.06.PM.mov