[lingfengyao]

Summary

With vad_model="fsmn-vad" and merge_vad=True (the SenseVoice demo/README default), merge_vad can return segments that contain no speech: the pause between two VAD segments, or [0, first_speech_start]. These are sent to the ASR model, and paraformer-zh decodes them into garbage (i the the the the the, 二 零 一 年 一 一 一 一 一, the alxxxxxxxxxxxxxxy YY) that ends up in the transcript.

from funasr.utils.vad_utils import merge_vad

merge_vad([[142870, 156260], [159720, 172040], [175490, 188040]], 15000)
# before: [[0, 142870], [142870, 156260], [156260, 159720], [159720, 172040], [172040, 175490], [175490, 188040]]
#          ^ silence                      ^ silence                          ^ silence
# after:  [[142870, 156260], [159720, 172040], [175490, 188040]]

Cause: segment starts and ends are sorted into one list, and the loop checks the distance to the next point but cuts at the current one. When consecutive speech segments are each close to max_length, it cuts at A.end and then at B.start, emitting the pause [A.end, B.start]. Because bg starts at 0, [0, first_start] can be emitted too.

Fix: after the existing merge loop, drop spans that overlap no VAD segment (one linear pass). The signature and return format are unchanged.

Type of change

• Bug fix

Validation

• python -m compileall funasr examples tests

• Added tests/test_merge_vad_silence.py. It covers the example above, leading silence, empty input, a single segment, a single segment longer than max_length, and randomized inputs (no pure-silence span; every VAD segment covered exactly once). On main, 3 of these tests fail; with this change all pass.

• The existing tests/test_merge_vad_length.py and the other vad / auto_model tests pass.

• End-to-end on long-form AISHELL-1 test: 20 speakers, 120 files of ~5 min (11.4 h) with random pauses between sentences. AutoModel(..., vad_model="fsmn-vad") on an A100, whole-file CER, 95% CI from a paired bootstrap over speakers.

model	merge_length_s	CER before	CER after	pure-silence segments before → after
paraformer-zh	15	3.50 [3.01, 4.02]	1.97 [1.74, 2.23]	98 → 0
paraformer-zh	20	2.05	1.95	4 → 0
SenseVoiceSmall	15	2.62	2.59	98 → 0
SenseVoiceSmall	20	2.53	2.53	4 → 0

For reference, VAD only without merging gives 1.96 (paraformer-zh) and 2.60 (SenseVoiceSmall).

User impact

This affects anyone using merge_vad=True on long audio, which is the default in the SenseVoice demo/README. The bug shows up when speech turns are close to merge_length_s, as in lectures, meetings and read speech. paraformer-zh users get hallucinated text inserted between sentences: 93 of the 98 silence spans above produced text, raising CER by 78%. SenseVoice mostly outputs <|nospeech|> on these spans, so it is only slightly affected, but it no longer wastes decoding on silence.

Notes for reviewers

• The output is identical to the current behaviour whenever no silent span is produced. On the 480 real fsmn-vad outputs from the evaluation, only the 148 that contained silent spans changed, so the existing test_merge_vad_length.py expectations still hold.
• I also tried a greedy whole-segment merge. It was only 0.03 pp better on paraformer-zh and no different on SenseVoice, so I kept the smaller, behaviour-preserving change.

[LauraGPT]

Reproduced on main 20f26b7c and checked head 8ee109bb.

• The new regression file fails on main (3 failed / 5 passed). On this head, test_merge_vad_silence.py plus test_merge_vad_length.py pass all 12 test methods, including the seeded randomized coverage checks.
• An independent L20Z check exercised the actual slice_padding_audio_samples with CUDA audio tensors and CUDA reductions in FP32 and BF16. The reported input produces three pure-silence slices on main in both dtypes. All 36 head cases (six interval layouts, three maximum lengths, two dtypes) preserve each speech sample exactly once and emit no pure-silence slices.
• This is bounded synthetic segmentation/slicing validation, not a pretrained ASR/CER evaluation. I have not independently verified the AISHELL accuracy table in the PR.

Two integration gates remain: this exact head currently has no hosted workflow/check run, and the existing full-project suite gate is unresolved because its online Conformer test requires unavailable cached weights. The current workflow path filters do not cover this VAD utility/new test. I am keeping this PR open; the focused CPU/CUDA results are not a full-suite or CI pass.

[lingfengyao]

Reproduced on main 20f26b7c and checked head 8ee109bb.

• The new regression file fails on main (3 failed / 5 passed). On this head, test_merge_vad_silence.py plus test_merge_vad_length.py pass all 12 test methods, including the seeded randomized coverage checks.
• An independent L20Z check exercised the actual slice_padding_audio_samples with CUDA audio tensors and CUDA reductions in FP32 and BF16. The reported input produces three pure-silence slices on main in both dtypes. All 36 head cases (six interval layouts, three maximum lengths, two dtypes) preserve each speech sample exactly once and emit no pure-silence slices.
• This is bounded synthetic segmentation/slicing validation, not a pretrained ASR/CER evaluation. I have not independently verified the AISHELL accuracy table in the PR.

Two integration gates remain: this exact head currently has no hosted workflow/check run, and the existing full-project suite gate is unresolved because its online Conformer test requires unavailable cached weights. The current workflow path filters do not cover this VAD utility/new test. I am keeping this PR open; the focused CPU/CUDA results are not a full-suite or CI pass.

Thanks for the review! Please let me know if there are any additional tests or changes you would like me to make to move this PR forward. I would be happy to help.