[tanishqbaweja]

Summary

Fixes #3870

With a timestamp export (4 outputs) and batch_size > 1, these wrappers pass each row of us_peaks to time_stamp_lfr6_onnx with the batch padding still attached:

• funasr_onnx: Paraformer.__call__ and ContextualParaformer.__call__ (SeacoParaformer inherits it);
• funasr_torch: Paraformer.__call__.

time_stamp_lfr6_onnx takes the frame count from the array length, and the last token ends halfway between its peak and that end. So the last token of every file shorter than the longest file in its batch ends past the end of its own audio.

The in-tree models already cut each row to its own length: us_peaks[i][: encoder_out_lens[i] * 3] in funasr/models/bicif_paraformer/model.py and funasr/models/seaco_paraformer/model.py. The C++ libtorch runtime (paraformer-torch.cpp) does the same with paraformer_length[index]*3. #3554 made the same per-sample trim for the main Paraformer. This PR does the same in the three Python call sites: zip feats_len into the loop and pass us_peaks_[: feat_len * 3] (2 lines per call site). feats_len is the LFR frame count fed to the model. As far as I can tell from the code, the in-tree encoder_out_lens equals it for these models, because a SANM encoder whose input_layer is pe adds a position encoding without subsampling. I did not check this against a released model config.

Two 440 Hz tones of 1 s (17 LFR frames) and 3 s (50), decoded by the real wrappers with only the model session replaced by BiCif-shaped outputs (tokens 你 好, CIF peaks inside each row's own frames, rows zero-padded like the exported graph):

wrapper	1 s file, batch_size=1	1 s file, batch_size=2 on main	1 s file, batch_size=2 with this PR
funasr_onnx Paraformer, ContextualParaformer, SeacoParaformer	[[70, 470], [470, 945]]	[[70, 470], [470, 1935]]	[[70, 470], [470, 945]]
funasr_torch Paraformer	[[70, 470], [470, 945]]	[[70, 470], [470, 1935]]	[[70, 470], [470, 945]]

On main, 好 ends at 1935 ms, 935 ms past the end of the 1000 ms file. The 3 s file gets [[70, 670], [1450, 2925]] in every case.

funasr_torch ContextualParaformer returns no timestamps, so it is not changed. In funasr_torch, feats_len is an int32 tensor, and slicing with a 0-d tensor works.

Type of change

• Bug fix
• Documentation
• Example or demo
• Runtime or deployment
• Benchmark or evaluation
• Model/training change

Validation

I added test_batched_timestamps_ignore_padding to tests/test_runtime_eval_native.py, for funasr_onnx Paraformer, ContextualParaformer and SeacoParaformer, and for funasr_torch Paraformer. It loads each wrapper with its real constructor, using the synthetic hotword-fixture graphs with lfr_m=7, lfr_n=6. It replaces only the acoustic model session (ort_infer, or ort_infer_bb for the hotword classes) with the outputs above. Then it checks that each file decoded alone has the timestamps in the table, and that one batch_size=2 call returns the same results.

test-runtime-eval.yml runs this file for changes under runtime/python/onnxruntime/funasr_onnx/** and runtime/python/libtorch/funasr_torch/**, so no workflow change is needed. With the same pins it installs (Python 3.12.15, torch 2.8.0+cpu, numpy 1.26.4, onnx 1.19.0, onnxruntime 1.23.2), on Windows 11:

python -m pytest -q tests/test_runtime_eval_cli.py tests/test_runtime_eval_native.py

• New tests on main source: 4 failed, 55 passed. All four fail at the batched call: [[70, 470], [470, 1935]] != [[70, 470], [470, 945]].
• This branch: 59 passed.

Also on this branch (Windows 11, Python 3.11.17, numpy 2.4.6, torch 2.10.0+cpu, onnxruntime 1.23.2):

• tests/test_runtime_eval_cli.py tests/test_runtime_eval_native.py tests/test_paraformer_timestamp_padding.py tests/test_paraformer_timestamp_contract.py tests/test_timestamp_tools.py: 71 passed on main (7651777), 75 passed on this branch.

• The funasr_onnx package test scripts overlap, empty_final and punctuation_empty_input pass from source.

• tests/test_funasr_torch_package.py: 13 passed, 1 failed, the same as on main. The failure is test_wheel_contains_all_runtime_sources, which on Windows builds its expected file names with backslashes.

• black --line-length=100 (24.4.0, as in pre-commit) leaves the test file unchanged.

• python -m compileall funasr examples tests

• Docs or links checked

• Runtime/deployment command tested (the wrapper runs above, CPU, with the model session replaced)

No real model was run. The model outputs are synthetic, so I have not compared these timestamps with a real export.

User impact

Users of a timestamp export (BiCif Paraformer such as paraformer-large-vad-punc, or SeACo Paraformer) who decode several files per call with batch_size > 1 through funasr_onnx Paraformer, ContextualParaformer, SeacoParaformer or funasr_torch Paraformer get the same timestamps as with batch_size=1. Before this change, the last token of each shorter file was stretched toward the end of the longest file in its batch. Results with batch_size=1 (the default), results without timestamps, and the text are not changed.

Notes for reviewers

• Only the timestamp loop in the two paraformer_bin.py files changes. timestamp_utils.py and postprocess_utils.py are not touched.
• git merge-tree --write-tree of this branch with the heads of fix(postprocess): keep a trailing @@ piece instead of dropping it #3836, fix(runtime): honor use_itn in funasr_onnx and funasr_torch SenseVoiceSmall #3860, fix(runtime): honor SenseVoice ITN and preserve mixed batch query tags #3861 and fix(runtime): merge spelled-out acronyms into one word in funasr_onnx and funasr_torch sentence_postprocess #3869 gives no conflicts.
• No version bump and no workflow change.