Skip to content

feat(rag): LanceDB retrieval, cited answers, and PDF fidelity fixes - #3

Merged
Rl0007 merged 40 commits into
mainfrom
feat/rag-chat-lancedb
Sep 9, 2026
Merged

Rl0007 merged 40 commits into
mainfrom
feat/rag-chat-lancedb

Conversation

@Rl0007

@Rl0007 Rl0007 commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

Implementing RAG based Search, you can select a project and converse on it.

Screen.Recording.2026-09-09.at.3.29.06.PM.mov

@greptile-apps

greptile-apps Bot commented Aug 11, 2026 •

Copy link
Copy Markdown

Too many files changed for review (135 files, 100 file limit).

Bypass the limit by tagging @greptile-apps to review.

Adds a retrieval layer and an Ask/RAG Lab UI over parsed documents, and fixes
several pipeline defects found while running two real ICAI study PDFs through it.

Retrieval (wikify/rag/)
- LanceDB store with local model2vec embeddings (256d, no API key, no server).
- Four modes: vector, full-text, hybrid (RRF), and filter. Filter returns EVERY
  match — "give me all X" is a metadata question, not a similarity guess.
- Contextual-retrieval prefix on embed text, parent-section expansion, optional
  LLM rerank that degrades to fusion order when unavailable.
- Permissions are a LanceDB pre-filter, never a post-hoc trim. An omitted ACL
  decision throws rather than searching everything.

Answering
- Intent router (exhaustive/semantic/hybrid) with follow-up rewriting; the
  decision and its plain-language reason are shown in the UI.
- Answers cite [n] markers; each citation carries a resolved page, line span and
  verbatim quote, verified against the source before display. Verification is two
  gates: fuzzy prose match AND exact equality on figures, signs and statutory
  refs — a flipped (+)/(-) previously passed a 0.85 similarity check.
- Honest refusal when nothing retrieved clears the bar.
- Per-question cost and token usage returned and displayed.
- Conversations persist to Wikify Ask Session/Message, so route decisions and
  citations become an evaluation set.

Pipeline fixes
- Sectioning stopped at the first heading longer than the 140-char title column,
  after the old tree was deleted — silently dropping 93.6% of a 236-page
  document. Titles are now clipped; coverage went 6.4% -> 99.8%.
- Page classification gated on `chars < 250 AND drawings > 40`, which no
  born-digital diagram page can satisfy; replaced with per-region detection
  (1/236 -> 231/236 pages routed correctly).
- Tables were being encoded as mermaid flowcharts, destroying row-to-rate
  correspondence; they now emit HTML tables, and diagrams are parsed and
  repaired (quoting node labels, `&` chains) before storage.
- Page verdicts were frozen at the baseline parse, so remediated pages still
  read "review"; they now track the adopted content, with a backfill patch.
- Remediation could adopt a near-empty candidate over a good one.
- Page edits now propagate into the sections and index built from them.

Frontend
- /ask and /rag-lab, with sources rendered before the answer, citation chips,
  provenance, and a naive-vs-routed comparison that shows what similarity search
  missed. Unranked results no longer render a meaningless full score bar.
- Mobile support across the app: shell navigation, drill-down replacements for
  desktop splits, and graph views that degrade to a list rather than an
  unreadable canvas.

Evaluation
- 12 golden questions with recall, precision and completeness, plus an HTML
  scorecard. Routed retrieval beats naive 66% -> 87-90% recall on the demo
  corpus; routing is an LLM call, so treat the figure as a range.
@Rl0007
Rl0007 force-pushed the feat/rag-chat-lancedb branch from 53ac237 to 59e6ce2 Compare August 11, 2026 03:20
Rl0007 and others added 27 commits August 11, 2026 09:01
Twelve images covering the Ask flow with citations, page/line provenance, the
cost meter, the naive-vs-routed comparison, the refusal state, a rendered
mermaid diagram, the reconstructed ICAI surcharge tables, the page verdict
list, and three mobile views plus dark mode.
Three defects found by testing the Ask path against two real ICAI documents,
plus the cleanup pass that followed.

Reranking — answers were being refused that the corpus covered
- `search()` re-sorted by fusion score after reranking and then applied the
  limit, so the reranked winner was pushed back to its fusion position and
  sliced off. "Slab rates under section 115BAC(1A)" scored 8.0 at fusion rank
  10 and never reached the answer; the question was refused. One `rank_key`
  now drives both the rerank and the slice.
- Grading 35 candidates in one call makes the classifier emit a run of zeros.
  Batching at 10 returns real scores. A batch whose reply skips candidates is
  dropped whole rather than letting the missing ones read as a confident 0.
- Refusal now needs two legs to agree: a below-floor rerank is overruled by a
  strong embedding match. Calibrated on both corpora — answerable questions
  peak 0.56-0.63, unanswerable 0.39-0.45. Genuine refusals still refuse.
- When the embedding leg overrules, citations carry `rerank_score: null`
  rather than 0.0, so the UI stops showing a dead relevance chip.

Cost — the reported figure was ~15x too low
Only the price printed on the response body was counted. The router and
reranker use the REST client, which returns cost inline; synthesis goes
through litellm, which reports price to a callback on its own thread. So the
meter added $0.00 for the leg carrying ~92% of the spend. A `CustomLogger`
keyed on `litellm_call_id` now bills each call back to the collector that
started it, and the second accounting mechanism in `history.py` is deleted —
session totals are summed from message rows so the header cannot drift.
Verified against OpenRouter's own generation records: an ICAI question costs
$0.047, not the $0.003 previously reported.

Efficiency
- Rerank batches run concurrently. `frappe.local` is unbound on a pool thread,
  so the API key is resolved on the calling thread and passed in. Measured
  19.81s -> 16.00s median per ask, with identical call count and spend.
- `index_status` did one LanceDB connect and one full scan per readable
  project; now one scan with a single `project IN (...)`.
- Page propagation rebuilt the whole FTS index once per section and re-read
  every page of the document each time; now one batched upsert and one FTS
  rebuild, loading only the relevant pages.
- Sectionising fired the reindex hook per row (~592 redis round trips, 295 of
  them no-ops); it now suspends per-row indexing and queues one rebuild.
- `get_page_image()` ran per page inside the remediation loop; the field is
  read with the rest of the row.
- Mermaid sources were parsed up to nine times per page; now once.
- `find_regions` read each page's drawings twice.

Reuse and dead code
Duplicate `get_pages_by_document` / batched-IN / page-line-span helpers
collapsed to one definition each; `useIsMobile` folded into `useMediaQuery` so
the app has one media-query registry; AskWiki reuses `MarkdownPreview`, which
also gives Ask answers mermaid rendering they never had; the eval harness now
calls production's `retrieve()` and `compare()` instead of copying them, which
is the drift it exists to catch. Removed an unreachable `EvalScoreboard`
branch, a dead `fts` relevance basis, an unused `page_regions` parameter, and
a `wait_for_prices` call that could add 5s to a failing ask without affecting
any reported number.

Streaming answers no longer re-parse the whole markdown per token.

References to five spec files that are not in this repo are rewritten to point
at the code that holds the information.
Asked "how many job descriptions are in the Demo Corpus project?", the agent
answered "there are none". There are 15. Two faults produced the same sentence,
so a correct chain of reasoning ended in a false statement.

- The `query` argument was a raw substring filter over title and hierarchy
  path. Questions are plural ("job descriptions"), titles are singular
  ("Job Description — Theatre Scrub Practitioner"), so every match was
  filtered away. It is now a narrowing hint: token-wise, normalised for case,
  punctuation and trailing plurals, and when it matches nothing the full set
  is returned with the filter reported as ignored rather than as absence.

- The model passed the project title, not its id. `Ctx.default_document`
  already guarded against exactly this, with a comment noting models echo
  display labels instead of bare ids; the project field had no such guard.
  Added `Ctx.default_project`, and closed the same hole in `semantic_search`.

Three outcomes that were indistinguishable now read differently: an unknown
section type (with the near matches, and an explicit note that this is not
evidence the content is missing), a valid type that is empty in scope (naming
the scope and which types do have content), and a filter that removed
everything (naming how many it removed). Every reply carries the in-scope
count and the resolved scope, so absence cannot be inferred from silence.

Also closes a fixture leak: types created by tests that drive the agent loop
survived rollback because the loop commits mid-turn, and were being offered to
users as real taxonomy suggestions.
OpenRouter routes by price by default and re-draws a provider per call, so
concurrent rerank batches were scattering across endpoints of different speed
and the wall clock was set by the slowest draw. Pinning the endpoint removes
the draw.

Measured over 5 batches, 9 runs each: unpinned 7.8s median with 1.9x effective
concurrency and a 24.7s worst case; pinned 2.6s median, 4.2x, 4.1s worst case.

The latency was the smaller half. Six of eighteen unpinned runs returned
truncated JSON, which discards the whole rerank verdict and drops the answer
back to fusion order — against zero of eighteen pinned. That is the most
likely source of the intermittent all-zero rerank we chased earlier, and it
means provider scatter was corrupting results, not just slowing them.

`allow_fallbacks` stays on: a reranker that cannot reach its preferred
provider must degrade to a slower one, never fail.

Retrieval with reranking measures 8.9s to 5.8s end to end. Total ask time is
unchanged in a single sample because the router leg drew slowly on that run;
the router change here is the same one-line pin, so that reading needs
repeating before anything is concluded from it.

Also raises the rerank worker cap now that batches no longer contend for a
provider draw.
test_ask_cost pins the per-ask figure against the sum of its individual LLM
calls, and the session total against the sum of its message rows. Without it
the 15x undercount could return silently: litellm reports synthesis price via
a callback rather than on the response, so the leg carrying ~92% of the spend
read as $0.00.

docs/rag-latency-prior-art.md records what production systems do about the
same problems, with sources and dates. Its main finding contradicts the
assumption we started from: synthesis is decode-bound, so trimming the 55k
character prompt is a cost fix rather than a speed fix, and the reranker's
model is the bottleneck rather than its depth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
`session` named two different things. The interface minted a per-ask correlation
token so it could pick its own deltas off the per-user realtime channel, and sent it
as the `Wikify Ask Session` docname the backend records turns against.

Two consequences. Every ask opened a new conversation, so a follow-up was never
rewritten against what came before it. And `recent_turns` ran `has_permission` on a
name that had never existed, which loads the document and raises DoesNotExistError —
the first non-Administrator to ask a question got a 404. Administrator short-circuits
the permission check, which is why this survived testing.

The token is now `stream` and is only echoed back on realtime payloads; `session` is
the docname, absent on the first ask and replayed from the response after that.
Switching project starts a new conversation rather than rewriting a follow-up against
turns about other documents.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
Page propagation hung off `Source Page.on_update`, so in production it almost never
ran: every real write to `canonical_markdown` goes through `store.set_canonical` or
`store.set_canonical_markdown`, and both use `frappe.db.set_value`, which fires no
doc_event. A re-remediated page was correct in Page Review while the sections built
from it — and every chunk indexed from those sections — kept serving the old text.
Only the tests exercised the hook, because they saved the document through the ORM.

Invalidation now hangs off `engine.store`, which already calls itself the write
funnel, and the tests drive it the same way. `events.suspended_indexing()` covers the
bulk passes in remediate and finalize, which write every page and then replace the
tree wholesale — invalidating page by page inside them would queue a redundant pass
over work that is about to be thrown away.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
The rebuild deleted a document's section tree and then inserted the new one with
nothing tying the two together. An insert that threw left the document holding
whatever prefix of the new tree had landed and none of the old — and jobs.remediate's
handler reverts the import to Review on the stated premise that the parse result is
intact, then commits, so the truncated tree became the document.

Delete and inserts now share a savepoint that is rolled back on failure, leaving the
previous tree standing. The over-length title that first exposed this is clipped, but
the hazard was the missing savepoint, not that one trigger.

The project rebuild also moves out of the finally block: on failure the old tree is
what the index already holds, so restoring it needs no rebuild.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
The gate re-implemented enough of mermaid's grammar in regex to decide whether a block
would parse, while the reader renders it with the real parser client-side. Every valid
construct the regex did not know was read as broken and the diagram discarded — `A & B
--> C` is valid mermaid 11 and cost page 6 its diagram twice before someone special-
cased it, which is the shape of the whole problem: the approximation is only ever
behind the grammar it is chasing.

Syntax now fails where the real parser lives; utils/mermaid.js already catches the
parse error and renders an error chip over the block's own source. What stays is the
judgment regex is actually good at and the reason this module exists: reading nodes
and edges to recognise a rate table flattened into two unbound columns, where a
confidently wrong rate is worse than no diagram.

Every block is now passed through the lossless quote repair unconditionally, since
deciding whether it was needed would mean judging syntax again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
… place

`set_canonical_markdown` replaces a page's text and scores nothing, but left the old
verdict and canonical composite on the row — so an agent edit could leave "pass 0.99"
sitting on content nobody had ever verified. The canonical-verdict fix only covered
`set_canonical`, the seam that carries a new composite, so this shape survived it.

Cleared rather than re-derived: scoring needs the page image and the baseline, which is
a verify pass, not something a write seam can do. Unscored reads as verdict "" and
composite 0 — the value `backfill_canonical_verdict` already skips, and the review UI
already renders an empty verdict as a gray dash and counts it as flagged.

The live caller is the agent's `edit_page_content`. `finalize_document` also writes
through this seam but has no production caller today.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
G1's naive leg was pinned at 5 correct from before the corpus was re-sectioned; it
measures 6 now, deterministically, across repeated runs. The PR's own headline already
quotes 6 correct and 9 missed, so the constant was the thing lagging, not the number.

Naive got better here, which narrows the gap the POC argues rather than breaking it:
routed still returns all 15 across all 5 documents. The two phrasings now agree at 6,
but stay asserted separately so a future divergence is visible rather than averaged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
The figure predates the coverage, sectioning and reranker fixes, and both failure modes
it turned on have been worked on since — so it is not a claim about the system as it
stands. Marked as stale in both directions: whether the grade moved is unmeasured.

Also records why it cannot simply be re-run: the twelve questions were asked by hand
and only the score survived, so there is nothing to reproduce or compare against. The
next grading should transcribe its question set the way rag/eval.py does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
Resolves NewImportDialog.vue in favour of main's batch PDF upload: this branch only
carried the older single-file FileUploader block, which that feature replaces, and the
rest of the file had already auto-merged to the batch design. The resolved file is
identical to main's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M1X9BSm7S3c1kR98fF9o59
Replaces the batched OpenRouter reranker with an in-process ONNX cross-encoder.
Scoring is total and deterministic, so the batching, the concurrency pool and the
provider pinning that existed only to stop the LLM returning short or all-zero
verdicts all go away. A rerank now costs no tokens and works on a site with no
LLM configured at all, where an absent key used to skip it silently.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
An exact-match cache keyed on the question, scope, history, model and the store's
write counter, so it can never outlive the corpus it answered from. A hit replays
the same realtime sequence the live path publishes, with zero spend and
`cached: True`; refusals are never stored, since they can come from a transient
failure rather than a fact about the corpus.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
The lab was a development surface; /ask is the product. With it gone the mobile
tab bar has room again, so the overflow sheet is back to user actions only and the
assistant returns to a floating button on desktop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
frappe-ui pins the panel to sm:min-h-[560px], which beats the height cap below a
688px viewport: the panel runs off-screen and clips its own header and Save button
with nothing to scroll them back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
`@frappe.whitelist()` with no `methods` allows GET, and Frappe runs
`validate_csrf_token` only on non-GET requests. So every state-changing and
billable endpoint was reachable from an `<img src=...>` on any page a logged-in
user visits: `rag.ask` and `rag.compare` spend real OpenRouter money and write
session rows as them, `rag.reindex` and the import triggers enqueue `timeout=3600`
long-queue jobs in a loop, and the whole of `sections.*` edits the tree.

26 endpoints across rag, imports, agent, projects and sections. The 16 that stay
GET are reads. `ask_history.delete_session` already had this and is the pattern.

No frontend change: frappe-ui's `call()` is unconditionally POST and sends the
CSRF header, and every `useCall` site for these methods already declared
`method: "POST"`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
`start_import`/`start_imports` wrote `file_url` verbatim to `Wikify Import.pdf`
and enqueued a worker to parse whatever it pointed at. Any user could name
another user's private attachment and have its contents re-published as wiki
sections; the batch endpoint multiplied that by MAX_BATCH per call.

Checked per File row rather than through `get_list`, because File's read rule
lives in a `has_permission` hook that a query filter never runs. The batch checks
every url before creating any Import, so one unreadable file enqueues nothing
rather than half a batch.

Fixtures now create real File rows (a one-page PDF via fitz, the house pattern
from test_parse_pipeline) since a url with no File behind it is no longer
importable. Verified red-green: the two batch guards fail with the check removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
`Wikify Ask Message` granted role All create + write + delete with `if_owner`,
and `if_owner` does not constrain `create`. Any logged-in user could insert a row
pointing `session` at a conversation they do not own, poisoning its transcript and
the cost totals `set_totals` sums from it.

Role All is now read-only `if_owner`. Turns are written by `history.append_message`
with `ignore_permissions`, which is justified: the conversation is the object of
authority and `record_turn` checks write on it first. The `on_trash` cascade uses
`frappe.db.delete`, so dropping delete costs it nothing — pinned by the existing
test_deleting_a_conversation_removes_its_turns.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
`remove_unverified_diagrams` returned "" for any block that failed the meaning
gate, destroying the model's structured read of the page with no way back. The
gate has a known false-positive class: MIN_VALUE_LEAVES = 3 trips on any fan of
three bare values, which on a rate referencer is most flowcharts.

The block is now demoted to a plain `text` fence with a one-line note, so the
reader sees what was meant and that it did not draw — which is what this module
already does for a diagram the renderer rejects on syntax. The source-crop
fallback is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
0.7 moved page invalidation onto the write funnel because `frappe.db.set_value`
fires no doc_event. The link below it was left open: `set_section_markdown` calls
itself THE write funnel for `Source Section.markdown` and writes the same way, so
the `Source Section` reindex hook never saw it either. An agent or UI edit changed
exactly the text the index serves and the index kept serving the old copy — the
bug this branch is named after, one link further down.

Two seams, both batched:

- `section_content_changed` reindexes the sections a write touched. Called from
  `set_section_markdown` and from `set_section_type`, since the type is a filter
  column every chunk carries.
- `document_structure_changed` queues one project rebuild from `_rebuild_tree`.
  `hierarchy_path` is embedded into each chunk as its contextual prefix, so a
  rename or a reparent invalidates the whole document rather than one row; the
  seven tree APIs that rebuild all route through there.

`toggle_include` is deliberately not wired: `include_in_wiki` is not a chunk field.

Verified red-green: the markdown-edit test fails with the funnel call removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
The branch pushes `readable_projects()` into the LanceDB `where` clause and makes
an omitted ACL decision throw. Two paths went around it.

`index_stats(projects=None)` defaulted to reading every project on the site — the
exact permissive default the AclDecision sentinels were introduced to make
impossible (see the note above ALL_PROJECTS). It now takes ACL_REQUIRED and calls
the same `assert_acl_decision` guard; `ALL_PROJECTS` is how a background caller
opts out.

`Ctx.default_project` resolved any Wikify Project by title with no permission
check, and the resolved id flows into the agent's read tools and `api/explore.py`,
which had no permission checks at all. So a prompt could widen the agent from its
attached project to any project on the site. Resolution now goes through
`get_list`, and Explore carries the same ACL as retrieval.

Explore subtracts unreadable projects rather than scoping to readable ones. A
document outside every project carries no permission statement and is visible
today; an allow-list would have hidden it as a side effect of a permission fix.

`readable_projects` / `assert_readable` move to `api/permission.py` so retrieval
and Explore cannot drift, and that module's docstring stops claiming the Wikify
DocTypes are System Manager-only — this PR ships two that are not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
…stion

Both retrieval models were pulled from the HF hub on first use, inside `search()`,
which `ask()` reaches on the web worker with `rerank=True` by default: the first
question after a deploy paid ~90MB of ONNX plus the embedder, once per worker
process, and an air-gapped bench hung on an HTTP timeout inside the answer path.
Neither download named a revision, so the hub could serve different weights
tomorrow — and the 0-10 rerank map, `answer.MIN_RERANK_SCORE` and every calibrated
score floor are measured against the logits these specific weights produce.

Both are now pinned to the commits the eval harness was measured on. model2vec's
`from_pretrained` takes no `revision` and defaults to `force_download=True`, so
the embedder resolves a `snapshot_download` and loads from the local path — which
also stops it re-fetching on every cold start.

`rag.warm.warm_models` runs on `after_migrate` and stands alone as a bench
command. It never fails the caller: a host with no network gets a logged error and
the failure surfaces at first use instead of breaking a migrate. A missing
embedder now throws with the command that fixes it rather than an HF traceback —
every mode except `filter` embeds the query, so there is nothing to degrade to.

Verified: warm_models returns both models loaded, and the 17 golden-question eval
tests pass against the pinned weights.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
…hooks

Four mechanical corrections from review.

`search()` took a bool named `rerank` while the module imports the `rerank`
module — the parameter shadowed it, and only the fact that `rerank_hits` resolves
the module from global scope kept it working. The next `rerank.scores(...)`
written inside `search()` would have raised AttributeError on a bool. Renamed to
`use_reranker`; the whitelisted `rerank` argument stays, since it is the wire
contract the SPA sends.

`api.rag.search` clamps `limit` to MAX_LIMIT. It widens the candidate list, and
with the reranker on each candidate is a 512-token cross-encoder forward pass on
the web worker inside the request — unbounded, an authenticated way to spend the
whole box on one call.

`replace_sections` uses `events.suspended_indexing()` rather than a second
hand-rolled copy of it, and the savepoint takes a random name the way Frappe's own
helper does, since re-declaring a name destroys the outer savepoint holding it.
The no-commit invariant the rollback depends on is now written down.

`hooks.py` drops 229 lines of `bench new-app` commented-out template; the live
statements are byte-identical. Frappe dependency narrowed to `<17.0.0` — v17 does
not exist and cannot have been tested against.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
Rl0007 and others added 5 commits September 9, 2026 08:29
The key carried the raw question plus the whole conversation history, so every turn
after the first was unique to its session and only a session's opening question
could ever hit — the opposite of the shared-syllabus traffic the module's docstring
justifies the cache with.

`ask` now routes before it builds the key and hands the decision to `answer()`, so
the key is the question the router RESOLVED: two readers whose follow-ups mean the
same thing share an entry, and two whose follow-ups differ still cannot collide.
`usage.collect()` nests, so moving the routing call out of `answer()` keeps every
leg of the turn in one total — and a cache hit now reports the routing call it
actually made instead of claiming to have cost nothing.

Two more holes in the same key:

- CACHE_VERSION. `index_version` retires an entry when the corpus moves, but
  nothing retired one when the synthesis prompt or a score floor moved, so a
  deploy kept serving answers written by the old behaviour for a full 24h TTL.
- A store that could not be opened returned version 0 — the same namespace a
  genuinely unindexed site uses — so an entry written during a blip outlived the
  corpus it answered from. It now yields no key at all, and the turn skips the
  cache.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
Prose was 30% of these files (1,132 of 3,801 lines). It is now 6%: every docstring
and every comment goes, except the ones that record a measurement, a workaround or
a gotcha — those survive as one-liners next to the code they explain.

Kept, because they are the reasons the code is shaped this way and cannot be
re-derived by reading it: the 512/320/256/128-token rerank recall figures, the
observed logit range the 0-10 map is built on, the RRF rank-based fusion scores
(0.0328 for both a perfect and a nonsense hit) and the vector floors calibrated
against them, the context-window trial that cut spend 55% and citations 12 -> 7,
the figure-comparison finding where "(+)" retyped as "(-)" scored 0.85 against a
0.82 floor, the `cache.get_value` stale-read that made coalescing swallow changes,
the doc_event that never fired, the savepoint's no-commit invariant, and every
`# ponytail:` marker.

Verified mechanically: for all twelve files the AST with docstrings removed is
identical before and after, so nothing but prose changed.

One test fix rides along, because the strip exposed it: the ask cost test asserted
against a live answer while the answer cache could serve it a previous run's entry
— it now bypasses the cache explicitly. It was latent before this change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
Re-picking the project on every visit was the most repeated action on the page.
The selection is now held in localStorage — per-browser and per-viewer by nature,
so it never travels between users, and a browser with site data blocked simply
doesn't remember it.

A remembered project the viewer can no longer see (deleted, or permissions
changed) is dropped as soon as the options load: `ask` refuses an unreadable
scope outright, so a stale selection would fail every question until they noticed
the select.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
An empty thread now opens on a centred composer under a time-aware greeting
rather than a dashed empty box; the first turn moves the composer to the
bottom, where a follow-up belongs. Sample questions sit under it and retire
with the landing.

The conversation id rides in the path (/ask/:conversationId), so a refresh,
a bookmark or a pasted link reopens the thread instead of dropping it.

The transcript only follows a streaming answer while the reader is already at
the bottom of it — scrolling up to re-read no longer gets yanked back down on
every delta.

The composer is extracted to AskComposer so its two placements cannot drift
apart, and a citation carries its import so a source card leads somewhere
before the wiki is generated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
Opening /ask/:conversationId painted the landing — greeting, composer,
suggestions — for the length of the fetch before the thread replaced it. That
flash reads as "new chat", the opposite of what the link said. The restore now
holds a turn-shaped skeleton instead.

The composer takes its placement as a prop rather than inferring it from the
turn count, which got the same case wrong: a restored thread's turns arrive
after the fetch, so the composer wore its landing face under a thread that was
already there.

Sample questions are now ones measured against the live index. Retrieval
applies a relevance floor and the previous set sat under it — "Summarise the
key sections" scored 2.8 and "What is still open or unresolved?" 1.7 against a
floor of 3.0, so both answered "not in this wiki", which is worse than offering
nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZAn49mkHhwX6e9uUDSYxr
@Rl0007
Rl0007 force-pushed the feat/rag-chat-lancedb branch from 6cee175 to 2745c8a Compare September 9, 2026 06:35
@Rl0007
Rl0007 merged commit 6bf5738 into main Sep 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant