Repository navigation
feat(cli): check-review findingFrames saves one plain frame per finding time - #6401
Conversation
The --skip-contrast-in-motion flag and HYPERFRAMES_CHECK_SKIP_CONTRAST_IN_MOTION env give way to a check-review capability in HF_TOOL_CONTEXT, so only a harness turns it on. The skill text and manifest go back to main.
…ng time With HF_TOOL_CONTEXT check-review.findingFrames, every error/warning finding in layout or contrast with a real bbox gets frameTime (the middle sample of its longest held run, by the heldMs gap rule) and frame (the absolute path of an unannotated full frame shared by every finding at that time). One seek and screenshot per distinct time, capped at 60 times with errors first, into the run's snapshot folder; not listed in snapshots, history or events. Without the capability, output is unchanged.
There was a problem hiding this comment.
Verdict: APPROVE · 0 blocker · 2 important · 0 nits
Selection/cap, the shared capture page's cleanup and the frame/frameTime metadata look correct. Two deadline edges in the frames phase are worth fixing before this goes to main; details inline.
Scope
Read the full diff at 9ad7914 against feat/check-review-capability, plus checkDeadline.ts, checkSnapshotDirectory.ts and studio-server snapshotRuns.ts (frame run retention: the newest 10 runs or 24 h, and unrecorded runs aren't protected, which matches the documented behavior). Ran the 5 focused files locally: 246 passed. tsc --noEmit is clean. No other reviews were on the PR. Unit-test CI doesn't run against the stacked base, so it needs to pass after the retarget.
The base branch was changed.
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
|
Follow-up stacked on this branch: #6409 adds |
Edit accuracy: accurate 271 (base branch 271), smooth 144 of thoseThe gate passes. Quarantined, measured but not gated (0) Unstable (2)
|
There was a problem hiding this comment.
APPROVE: 0 open, 0 new · 5e14d8b → 0aacda3 (main integration). The diff against main matches the patch approved at 5e14d8b, except for the checkReview() doc comment that #6400 removed on main. Main changed no other file this PR touches since the old base. 249 focused tests pass locally and the CLI typecheck is clean. Both CI runs and Windows render verification pass on this head.
What
A second
check-reviewfield inHF_TOOL_CONTEXT,findingFrames. When it is on,checksaves one plain,unannotated full frame for each distinct finding time, and adds two fields to each finding it covers:
frameTime: the time the frame was taken.frame: the absolute path of that PNG. Findings at the sameframeTimeshare one file.Which findings get a frame:
errorandwarningfindings in thelayoutandcontrastsections that have a realbbox.motionandcraftfindings,infofindings and findings without a bbox get no frame and no new fields.Without the capability, output is unchanged.
Based on main. #6400 merged as
257bf285fon 2026-10-26 at 07:58:22 UTC. Landed main was merged normally into this branch in signed commit0aacda324; the diff is onlyfindingFrames.Why
An agent reviewing check output gets a bbox and a time, but has to run
snapshotitself to see what the findinglooks like, often at a time that does not show the problem. The existing annotated snapshots draw over the frame
and are taken at fixed sample times. A plain frame at a time when the finding is actually present lets a harness
crop the bbox or show the whole frame to a model without another command.
Related work
Builds on merged #6400, which adds the
check-reviewcapability. Builds on #6396.How
times, where two samplesmore than 1000 ms apart start a new run. This is the same gap rule
heldMsuses, so the frame comes from thestretch where the finding persists longest rather than from a one-sample blip (
checkFindingFrames.ts).frameTime. It reuses the post-report capture path that--snapshotsfinding crops already use(
withFindingCapturePageincheckBrowser.ts, now shared by crops and frames). When nothing qualifies, nosession is launched.
time falls past the cap get no
frame.snapshots, not recorded in projecthistory, and not reported as artifact
events.check-reviewtable indocs/developers/harness-tool-context.mdxgets afindingFramesrow.Latency, wall clock of
check --json, frames on vs this branch's base, in seconds:That is +0.8 to 1.3 s when any finding qualifies. Most of it is the second browser launch (0.6 to 0.9 s); the
seeks and screenshots are the rest. A run with no qualifying finding costs nothing.
Test plan
check.test.tscoversselectFindingFrames(layout and contrast errors and warnings witha real bbox only, one frame per time; the cap keeps errors before warnings),
middleOfLongestRun, turningframes on only through
HF_TOOL_CONTEXT, and addingframe/frameTimeonly when on without touchingsnapshots.toolContext.test.tscovers parsingfindingFrames.vitest run src/commands/check.test.ts src/utils/toolContext.test.ts src/utils/layoutAudit.test.ts src/utils/checkBrowser.test.ts src/commands/checkArtifacts.test.tsin
packages/cli: 7 files, 259 tests passed after main integration at0aacda324; includessrc/utils/checkSnapshotDirectory.test.tsandsrc/utils/checkDeadline.test.ts. New regressions for invalid time selection and cancellation both failed before their fixes. Earlier-stop deduplication (snapshots on/off) and a progressing 160-second fake-clock batch also fail before their fixes and pass after. Capture tests verify writes and cleanup with mocked browser sessions, including a later screenshot failure.tsc --noEmitinpackages/cli: clean.oxlintandoxfmt --checkon changed TypeScript: clean. Comment citation and density checks pass against main. Initial workspace build passed using existing dependencies.check --jsonon four real projects with the capability on. It was measuredon the pre-squash branch, whose patch is identical to this PR's first commit; the second commit only renames a
flag in two doc comments.
Not covered
bboxis measured at the finding's anchortime, not atframeTime. For an element that moves between thetwo, the box can sit somewhere else on the frame. This is documented in the field description.
motionorcraftfindings.