[David-Wu1119]

Summary

rich_transcription_postprocess() glues the last word of one VAD segment to the first word of the next for English (and Korean) SenseVoice output.

AutoModel.inference_with_vad joins the tagged text of each VAD segment with a single space, so a long recording reaches the postprocessor as several <|lang|> blocks:

<|en|><|NEUTRAL|><|Speech|><|withitn|>Hello world. <|en|><|NEUTRAL|><|Speech|><|withitn|>How are you?

The function strips every block and concatenates them with no separator:

from funasr.utils.postprocess_utils import rich_transcription_postprocess
rich_transcription_postprocess(
    "<|en|><|NEUTRAL|><|Speech|><|withitn|>Hello world. "
    "<|en|><|NEUTRAL|><|Speech|><|withitn|>How are you?"
)
# main:   'Hello world.How are you?'
# branch: 'Hello world. How are you?'

This is the README / SenseVoice example flow (vad_model="fsmn-vad" followed by rich_transcription_postprocess(res[0]["text"])) on any English recording long enough to produce more than one segment (with or without merge_vad).

Fix: when appending a block, insert one space if both boundary characters belong to a space-separated script (ASCII or Hangul). Chinese/Japanese text and emoji boundaries are joined exactly as before. The same change is applied to the funasr_onnx and funasr_torch copies so the three shipped postprocessors stay identical (the existing parity tests load all three).

Type of change

• Bug fix
• Documentation
• Example or demo
• Runtime or deployment
• Benchmark or evaluation
• Model/training change

Validation

Environment: macOS arm64, Python 3.11.15, CPU-only torch 2.15.0, pytest 9.2.0.

• New parametrized test test_vad_segments_keep_a_space_between_spaced_scripts in tests/test_rich_transcription_postprocess.py (English, mid-sentence split, Korean, plus Chinese and Chinese-to-English controls, for all three copies).
• test_repeated_event_only_segments_keep_following_text in tests/test_rich_transcription_event_segments.py pinned the glued "helloworld" output for two separate <|en|> blocks; it now expects "hello world".
• python -m pytest -q tests/test_rich_transcription_postprocess.py tests/test_rich_transcription_event_segments.py: with the main source files, 18 failed / 33 passed (9 new spaced-script cases + the 9 updated event cases; the CJK controls pass); with this change, 51 passed.
• Text postprocessing CI set plus neighbours (test_postprocess_hotwords.py, test_sentence_postprocess_timestamps.py, test_speaker_emotion_docs.py, test_paraformer_timestamp_*.py, test_sensevoice_word_timestamps.py, test_funasr_onnx_installed.py, test_funasr_torch_package.py): 143 passed, 1 skipped.
• The join was also checked through the real AutoModel.inference_with_vad with stub VAD/ASR models returning two English segments: main prints Hello world.How are you?, this branch Hello world. How are you?.
• python -m compileall funasr examples tests
• Docs or links checked
• Runtime/deployment command tested

No model weights were run; SenseVoice output was simulated with tagged strings.

User impact

SenseVoice users transcribing English or Korean audio longer than one VAD segment get readable text instead of words running into each other at every segment boundary.

Notes for reviewers

• Korean is included because it separates words with spaces; drop the Hangul range if you prefer an ASCII-only rule.
• A boundary where an emotion/event emoji sits between the two blocks is still joined without a space, matching how format_str_v2 already glues emoji to adjacent text.

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.