Repository navigation
Fix runtime hotword bounds on current main (preserves #3882) - #3885
Merged
Merged
Conversation
ContextualParaformer.proc_hotword in funasr_onnx and funasr_torch
(SeacoParaformer inherits it) split the hotword string with split(" ")
and padded each hotword into a row of 10 ids. A hotword longer than 10
characters did not fit, so the whole call raised "ValueError: could not
broadcast input array from shape (12,) into shape (10,)". Repeated,
leading or trailing spaces, and an empty hotword string, gave empty
hotwords: rows of padding whose embedding was read at position -1 and
passed to the model as extra hotwords.
The exported embedding model takes exactly 10 ids per hotword (only
num_hotwords is dynamic in ContextualEmbedderExport), and the C++
runtime (CompileHotwordEmbedding in paraformer.cpp and
paraformer-torch.cpp) keeps the first max_hotword_len = 10 tokens of
each hotword and skips empty ones. Do the same: split on whitespace, as
generate_hotwords_list in funasr/models does, and keep the first 10
characters of each hotword.
Extend tests/test_runtime_eval_native.py: for both classes in both
packages, built from the hotword fixture graphs, check the hotword ids
and embedding positions for a string with extra spaces and a
12-character hotword and for an empty string, and that both strings
decode every file.
Retain the original #3882 author commit and carry forward the exact accepted 1.4.26 main tree. Only the original three-file hotword patch differs from main. Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Integrate the original #3882 hotword fix on the current accepted 1.4.26 main, retaining the original author's commit as the parent. Relative to current main this changes only the original three files: the ONNX and Torch hotword preprocessing and its native-runtime tests. The original fork branch is untouched. Refs #3881.
The exported embedding input is ten characters wide. The fix keeps the supported prefix of longer words, computes positions after truncation, and drops empty entries produced by whitespace.
Verification
This does not publish standalone ONNX or Torch distributions, and the already published root 1.4.26 does not contain this later runtime change. No production service or model-weight download was performed.