[xuanruli]

What

hyperframes transcribe <9.5-min Mandarin mp4> --engine whisper --model small --language zh --json reported wordCount: 2: transcript.json held two entries of ~280 s each, covering the whole speech. Captions and audio --read (which builds sentences from the transcript) had nothing to split.

Why

parseWhisperCpp merges any token without a leading space into the previous word, since whisper.cpp marks word boundaries with one. Chinese and Japanese tokens never carry a leading space, not even at the start of a segment, so the whole transcript collapsed across segments. The whisper-cli args and the token loop in transcribe.ts are fine (the reported wordCount comes from loadTranscript).

Change

• A token that crosses a CJK boundary starts its own word, so 用 + AI + 剪 stay separate; full-width punctuation (，。！？、 …) still attaches to the word before it.
• Cues also end at 。！？, not only .!?.
• The merge decision moves into continuesWord to keep parseWhisperCpp under the fallow gate.

Thai/Lao/Khmer are unchanged: their tokens are sub-word fragments, and joinTokens would re-space them.

CI repair

The inherited support-CI failure parsed the version range as the package name in Bun’s minimum-release-age error. Capture the package name so the existing scoped exclusion and retry work as intended.

Author verification

• Synthetic Mandarin speech (macOS say -v Tingting, no customer footage) through real whisper-cli + ggml-small, then loadTranscript: 1 word before → 48 per-token timed words after; a rare-character sentence gives 30.
• New whisper-cpp CJK tokens tests in normalize.test.ts use a synthetic whisper JSON fixture; all 3 fail on main and pass here (79/79 in the file).
• src/whisper + commands/transcribe.test.ts: 283 pass; 4 failures in sherpa.test.ts / manager.install.test.ts fail identically on unmodified main on this machine (a real runtime is installed).
• tsc, oxlint, oxfmt, fallow pass.

Not covered: live progress words (segmentWords in transcribe.ts) still split on whitespace, so a CJK segment shows as one interim word until the final transcript replaces it.

Followthrough verification

• Independently ran normalize.test.ts: 79/79 pass using existing dependencies/generated artifacts.
• Verified installConsumer with a mocked child process: writes fake-dep, not ^1.1.0, to the exclusion list and retries. No real package install was run locally.
• Repair lint, format, and commit hooks pass. Fresh CI validates the real install fixture.

[jerrai-bot-heygen]

Reviewed exact head f9c6ba35739df899ffa00f5d744e4d65860d5cea. The Whisper parser now separates adjacent CJK tokens while retaining their individual start/end times, still joins punctuation to its preceding token, and makes full-width sentence marks end cues. The mixed-script fixture asserts timed words and cue text; the minimum-release-age regex now captures the package specifier rather than the version range. I found no blocking regression in these paths. Exact-head Lint, Typecheck, Build, CLI smoke, and Test (support) succeeded; the broader runtime test job was still running at review. I did not run Whisper on real Mandarin audio or reproduce the author's normalization test locally.

— Jerrai

[miguel-heygen]

Additive independent review at f9c6ba35739df899ffa00f5d744e4d65860d5cea: no additional findings beyond Jerrai’s review.

normalize.ts:130–150 keeps punctuation/contractions attached while separating CJK token boundaries; normalize.ts:340 recognizes full-width sentence endings. Independent production-module controls pass for 7 mixed-script/legacy token cases (Chinese/Latin, Japanese, contractions, Thai and Korean) and 6 full-width sentence/closing-quote cases. I also ran 62 existing normalization tests, including all 3 new regressions, against the exact source snapshot: all pass. The full 79-test suite could not collect locally because happy-dom is absent; the local run excluded HTML-patching and speech-onset groups rather than installing dependencies.

I inspected completed support job 119771647739: the real local-registry fixture passes the old-version selection, same-day-only version retry, and missing-package failure cases; its retry output names fake-dep, confirming the regex captures the package rather than the range. Typecheck, lint, format, build and CLI smoke pass. Broader runtime/integration/Windows jobs were still running in my last snapshot; this is not an all-CI-green claim.

No real Whisper model or audio run by me; author’s Mandarin evidence remains attributed. No source changes or installs.

Verdict: COMMENT — source-clear, no additional blocker.
Reasoning: the targeted production controls and existing normalization tests preserve timing/legacy behavior, and the completed support fixture exercises the regex repair. Jerrai’s exact-head approval already covers the merge verdict.

— Magi Home