[tanishqbaweja]

Summary

Fixes #3868

In the funasr_onnx and funasr_torch copies of postprocess_utils.py, abbr_dispose gives a run of single-letter tokens one [begin, end] timestamp, but still appends every letter as its own word. In 2023, 2ab8b7f (#270) changed the in-tree copy to build the acronym as one word (abbr_word) and append it once. That change only went into funasr/utils/postprocess_utils.py.

So with a timestamp model, each extra letter adds a word with no timestamp. Tokens 我 用 a i 写 代 码 with one timestamp per token ([i*100, (i+1)*100]):

copy	text	words	timestamps
funasr/utils	我 用 AI 写 代 码	6	6
funasr_onnx, funasr_torch on main	我 用 A I 写 代 码	7	6
funasr_onnx, funasr_torch with this PR	我 用 AI 写 代 码	6	6

i b m is here gives I B M is here (5 words, 3 timestamps) and c c t v gives C C T V (4 words, 1 timestamp). The untimed text is already 我用AI写代码, but the returned word list is split the same way.

The Paraformer wrappers return preds, timestamp and the unmerged raw_tokens, so preds.split() is the only word list that can be paired with timestamp. I ran the real funasr_onnx Paraformer.__call__ with only infer replaced by BiCif-shaped outputs (am scores for these tokens, token_num, us_alphas, us_peaks) on a 2 s tone. On main it returns preds='我 用 A I 写 代 码' and 6 timestamps: I gets 写's span, 写 gets 代's, 代 gets 码's, and 码 gets none. funasr_torch Paraformer (with ort_infer replaced) gives the same result. With this PR both give 我 用 AI 写 代 码 and every word keeps its own time.

The fix is the in-tree code, in both runtime copies: build abbr_word and append it once (4 insertions, 3 deletions per file). After it, the runtime abbr_dispose differs from the in-tree one only by its docstring and the bounds checks from #2122/#2209. Those are not needed here, because these copies raise ValueError for a token they do not recognise, so every word they return has a timestamp.

tests/test_sentence_postprocess_timestamps.py already checks the three copies against one expectation. I added two tests at the end of the class:

• 我 用 a i 写 代 码 and i b m is here, with timestamps;
• 看 c c t v 新 闻, without timestamps (("看CCTV新闻", ["看", "CCTV", "新", "闻"])).

Type of change

• Bug fix
• Documentation
• Example or demo
• Runtime or deployment
• Benchmark or evaluation
• Model/training change

Validation

test-text-postprocessing.yml set, with the pins it installs (pytest 8.1.1, pypinyin 0.55.0, rapidfuzz 3.14.5), on Python 3.10.22 and 3.12.15:

python -m pytest -q tests/test_postprocess_hotwords.py tests/test_rich_transcription_postprocess.py tests/test_sentence_postprocess_timestamps.py

• main (7651777): 30 passed.
• New tests on main source: 2 failed, 30 passed. All 6 failing subtests are the funasr_onnx and funasr_torch copies.
• This branch: 32 passed.

I also compared the three copies on 20,000 random token sequences (Chinese, single letters, words, @@ pieces, <unk>, <s>, </s>), with and without timestamps. On main, a runtime copy gives a different result from the in-tree copy for thousands of them. With this PR, none differ.

Also on this branch (Windows 11, Python 3.11.17, numpy 2.4.6, onnxruntime 1.23.2, torch 2.10.0+cpu):

• tests/test_runtime_eval_cli.py tests/test_runtime_eval_native.py: 55 passed.

• tests/test_paraformer_timestamp_contract.py, tests/test_paraformer_timestamp_padding.py and both rich transcription test files: all pass.

• The funasr_onnx package test scripts (overlap, empty_final, punctuation_empty_input) pass from source.

• tests/test_funasr_torch_package.py: 13 passed, 1 failed. The failure is test_wheel_contains_all_runtime_sources, and it fails on main too, because on Windows the test builds its expected file names with backslashes (str(Path.relative_to)).

• python -m compileall funasr examples tests

• Docs or links checked

• Runtime/deployment command tested (the wrapper runs above, with the model session replaced)

No real model was run, so I have not seen a model emit these tokens. abbr_dispose exists to merge runs of single-letter tokens, and the C++ runtime's vocab.cpp also joins consecutive single letters without a space. #1030 describes output such as 三角形ABC and 力FA, the upper-case form this merge produces.

User impact

Users of a timestamp export (4 outputs: BiCif Paraformer such as paraformer-large-vad-punc, or SeACo Paraformer) through funasr_onnx Paraformer and SeacoParaformer or funasr_torch Paraformer get one word per timestamp again when the output contains a spelled-out acronym. The untimed word lists of these wrappers and of the funasr_onnx streaming Paraformer also keep the acronym as one word. The returned text without timestamps does not change.

Notes for reviewers

• Only abbr_dispose in the two runtime copies changes. funasr/utils/postprocess_utils.py is not touched.
• fix(postprocess): keep a trailing @@ piece instead of dropping it #3836 edits the end of sentence_postprocess in the same files and adds tests to the same test file. The hunks do not overlap: git merge-tree --write-tree of this branch and fix(postprocess): keep a trailing @@ piece instead of dropping it #3836's head (84c7b76) gives no conflicts.
• No version bump, and no workflow change: test-text-postprocessing.yml already runs this test file for these paths.