auto_model 中punc模型入参为空触发的bug #1660

🐛 Bug

punc入参的文本为空时报错如下：

Expected tensor for argument #1 'indices' to have one of the following scalar types: Long, Int; but got torch.cuda.DoubleTensor instead (while checking arguments for embedding)

To Reproduce

Steps to reproduce the behavior (always include the command you ran):
1.在使用pipeline调用speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch模型进行asr推理，当asr识别结果为空时（没有识别有人说话），组合中的punc模型对asr的空文本结果进行推理，导致报错。

pipeline(task=Tasks.auto_speech_recognition, model='iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch', model_revision="v2.0.4", vad_model='iic/speech_fsmn_vad_zh-cn-16k-common-pytorch', vad_model_revision="v2.0.4", punc_model='iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch', punc_model_revision="v2.0.4", spk_model="iic/speech_campplus_sv_zh-cn_16k-common", spk_model_revision="v2.0.2", spk_mode='punc_segment', )

2. See error

Code sample

File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/modelscope/pipelines/audio/funasr_pipeline.py", line 73, in call
output = self.model(*args, **kwargs)
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/modelscope/models/base/base_model.py", line 35, in call
return self.postprocess(self.forward(*args, **kwargs))
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/modelscope/models/audio/funasr/model.py", line 61, in forward
output = self.model.generate(*args, **kwargs)
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/funasr/auto/auto_model.py", line 205, in generate
return self.inference_with_vad(input, input_len=input_len, **cfg)
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/funasr/auto/auto_model.py", line 386, in inference_with_vad
punc_res = self.inference(result["text"], model=self.punc_model, kwargs=self.punc_kwargs, disable_pbar=True, **cfg)
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/funasr/auto/auto_model.py", line 237, in inference
results, meta_data = model.inference(**batch, **kwargs)
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/funasr/models/ct_transformer/model.py", line 272, in inference
y, _ = self.punc_forward(**data)
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/funasr/models/ct_transformer/model.py", line 83, in punc_forward
x = self.embed(text)
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/torch/nn/modules/sparse.py", line 162, in forward
return F.embedding(
File "/root/miniconda3/envs/funasr/lib/python3.8/site-packages/torch/nn/functional.py", line 2233, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected tensor for argument #1 'indices' to have one of the following scalar types: Long, Int; but got torch.cuda.DoubleTensor instead (while checking arguments for embedding)

Expected behavior

纠正这个bug，如果已纠正，请告知正确的funasr版本

Environment

• Linux: Alibaba Cloud Linux 3.2104 LTS 64位
• FunASR Version :1.0.8
• ModelScope Version :1.11.1
• PyTorch Version :2.1.2
• pip install funasr
• Python version : 3.8
• GPU (NVIDIA T4)
• CUDA/cuDNN version : NVIDIA-SMI 530.30.02 Driver Version: 530.30.02 CUDA Version: 12.1