Repository navigation
fix(images): name an unread image's bytes in the placeholder - #783
Merged
Merged
Conversation
An image that no model can read becomes a short placeholder, and that was all the Bot got: it could not open the picture, and the content-addressed bytes sat on disk unmentioned. Reproduced with a picture in Bot chat on a tree with no vision model: the request carried "[Image ...: it could not be read right now.]" and the warning "could not read image sha256:...: no vision model is available", while /data/.../attachments/v1/objects/bd/bdc... was 213 KB on the same machine. `imagePlaceholder()` is now the one placeholder for both places an image becomes text (a Bot chat through `imageAsText`, the group relay through `imagesForGroup`), and it names the file when the Bot can open it. `attachmentFilePath()` resolves it the way the rest of the core reaches files: the attachment store's host path goes through the file sandbox's mapping (`processPathFromHostPath`), and the local object layout (`<home>/attachments/v1/objects/<first two>/<sha256>`) is the fallback when the store or the mapping is absent. Either way `stat` must succeed and the path must be a file: a placeholder that points at nothing is worse than no path, and an image id that is not content-addressed gets none. Pinned by #779: the object layout and its refusal of non-sha256 ids; the file exists / is a directory / is missing cases; the store-and-mapping path and the fallback; the three placeholder forms; and, in the Session harness with every vision route disabled, that an unread image names its bytes both in a group relay and in a Bot chat, and stays short when the bytes are gone. Drop the path from the placeholder and images+session-identity is 10/12 with the two path assertions red; drop the `stat` check and it is 9/12 with the three "must exist" assertions red. With the fix: 12/12 in those two files, and `patches/ds-bot` regenerates to the same `core/` (applycheck: 114 patched files, 331 compared, MATCH). Part of #779.
`core/src/host/images.js` is an owned path of the `core-host` subsystem, so the arch-docs gate fails a change to it without an update to `docs/subsystems/core-host.md`. This adds the missing subsection: which model reads an image (`ocrRoute`, `ocrModel`, `textOnlyModels`), that the reading is cached in `<home>/ocr.json`, and the rule the placeholder follows — one placeholder for a Bot chat and the group relay, the file named only when the Bot's own tools can open it (the store's `imageHostPath` through `processPathFromHostPath`, else the local object layout, either way `stat`-checked), nothing copied into the working folder. No behaviour change; the code and its tests are unchanged at this tip. Part of #779.
Collaborator
Author
|
Verdict: PASS at 865c84e Ran (head
What I checked:
Not run: a real vision model (this host has none — that is Non-blocking:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Written-by: Dev
Tip SHA: 865c84e
Fixes #779
What changed
core/src/host/images.js:imagePlaceholder()is now the single placeholder for both places an image turns into text, and it names the file when the Bot can open it.attachmentFilePath()resolves the attachment store's host path through the file sandbox's mapping (processPathFromHostPath) and falls back to the local object layout (<home>/attachments/v1/objects/<first two>/<sha256>); either candidate muststatas a real file, and an image id that is notsha256:<64 hex>gets no path at all.imageAsText(Bot chat) andimagesForGroup(group relay) both pass the path through it, and only look it up when the reading failed.core/test/images.test.js: the object layout and its refusal of other ids; a file that exists is named, a directory and a missing file are not; the store path wins when the sandbox maps it and the object layout is the fallback; the three placeholder forms.core/test/session-identity.test.js: the harness takes runtime config and returns its home; with every vision route disabled (textOnlyModels), an unread image names its bytes in a group relay and in a Bot chat alike, no vision request goes out, and an image whose bytes are gone keeps the short placeholder.docs/subsystems/core-host.md: the image subsection its arch-docs ownership asks for (route and cache, and the placeholder rule);core/README.mdandREADME.md: images are read by a model that declares image input (the configuredocrModelfirst), and the placeholder names the file when no model can read it. No configuration is changed by this PR:cordis.patch.ymlstays as it is.patches/ds-bot/:src/host/images.js,test/images.test.jsandtest/session-identity.test.jsjoined0001-core-fixesintopics.json, and the patch was regenerated fromcore/.What was wrong
A picture no model can read reaches the Bot as
[Image "…" (…): you cannot see images and it could not be read right now. Ask the user what it shows if it matters.], while the bytes sit in the attachment store on the same machine — on this box/data/dsh/attachments/v1/objects/bd/bdc77c70…, 213569 bytes, with the log linecould not read image sha256:bdc77c70…: no vision model is available. Nothing in the placeholder says where the file is, so a Bot that could open it with its own file tools asks the user to describe the picture instead.What I ran
At this tip (base
d83d762d, Node 24.10.0 + DSH0.2.0-rc.2from npm,TZ=UTC):core:node --import ./test/harness-resolve.mjs --test test/images.test.js test/session-identity.test.js→ 12/12.node .github/scripts/changelog.mjs check→ 38 fragments, all well formed.patches/ds-bot/files/+ all patches applied tods-bot/reproducescore/(114 patched files,MATCH 331 files).Red without the change, each cut alone at this tip:
imagePlaceholder()→ 10/12: red on the placeholder names the file, its size and its type; without a path it stays as it was and an unread image names its bytes in a group and in a chat alike — the two assertions that carry the fix.statcheck replaced by "any string is a path" → 9/12: red on a file the Bot can open is named, and one that is not there is never invented, the attachment store's path wins when the file sandbox can map it (its missing-file half) and an unread image whose bytes are gone keeps the placeholder short.Not run on this box:
npm run lint(no oxlint here),scripts/ds-bot-patches.mjs checkandarch-docs(both needgit), smoke and Desktop. CI runs them at this tip.Risk
T3 (
core/src/host/**). Narrow: the placeholder text grows by one sentence and only when a readable file exists; the existing reading, caching and offload paths are unchanged, and a failed lookup returns the placeholder exactly as before.