Repository navigation
Description
load_data in runtime/python/onnxruntime/funasr_onnx sends every path that does not end in .wav through pydub, writes the result to <name>.wav next to the input, and then loads <name>.wav. This affects Paraformer (also used by ContextualParaformer and SeacoParaformer), the streaming Paraformer, Fsmn_vad and SenseVoiceSmall:
def load_wav(path: str) -> np.ndarray:
if not path.lower().endswith('.wav'):
import os
input_path = path
path = os.path.splitext(path)[0]+'.wav'
convert_to_wav(input_path,path) #将mp3格式转换成wav格式
waveform, _ = librosa.load(path, sr=fs)
return waveformconvert_to_wav imports pydub, runs AudioSegment.from_mp3(input_path).export(output_path, format="wav") and only prints the exception if that fails.
- pydub is not a dependency of funasr-onnx (or of funasr), so on a clean install every .mp3, .flac or .ogg path raises
ModuleNotFoundError: No module named 'pydub'. That includes theexample/en.mp3input ofdemo_sencevoice_small.py. - With pydub but no ffmpeg, the conversion prints
转换失败:...andlibrosa.loadthen raisesFileNotFoundErrorfor<name>.wav, a file that was never passed. If a different<name>.wavsits next to the input, that recording is loaded instead, with no error. - With pydub and ffmpeg,
from_mp3forces the MP3 demuxer, so.flac,.oggand.m4apaths still fail the same way (ffmpeg:Invalid data found when processing input). Only.mp3converts, and its export overwrites any existing<name>.wavnext to the input.
librosa, already a dependency, reads these files directly, and funasr_torch and Fsmn_vad_online already load every path with plain librosa.load(path, sr=fs).
Repro on main (20f26b7), no model files needed, with runtime/python/onnxruntime on PYTHONPATH:
import sys
import numpy as np
import soundfile
from funasr_onnx import Paraformer
tone = (np.sin(np.arange(48000) * 2 * np.pi * 440 / 16000) * 0.1).astype(np.float32) # 3 s
soundfile.write("talk.flac", tone, 16000)
if len(sys.argv) > 1: # a different 1 s recording with the same name
soundfile.write("talk.wav", np.zeros(16000, np.float32), 16000)
model = Paraformer.__new__(Paraformer) # load_data uses no model state
print(len(model.load_data("talk.flac", 16000)[0]))- Without pydub:
ModuleNotFoundError: No module named 'pydub'. - With pydub 0.25.1 and no ffmpeg:
转换失败:[WinError 2] The system cannot find the file specified, thenFileNotFoundError: [Errno 2] No such file or directory: 'talk.wav'. - The same, run with an argument (a 1 s
talk.wavnext totalk.flac):转换失败:..., then16000.
Expected: 48000, which is what librosa.load("talk.flac", sr=16000) returns. .ogg and .mp3 files (soundfile writes and reads both here) give the same results, and so does load_data of the other wrappers listed above.
Environment: Windows 11, Python 3.11.17, librosa 0.11.0, soundfile 0.14.0 (libsndfile 1.2.2), audioread 3.1.0, no ffmpeg on PATH except for the pydub-and-ffmpeg check above (ffmpeg 8.1.3); funasr_onnx from runtime/python/onnxruntime on main. A funasr-onnx package installed from main into a fresh Python 3.12 venv (numpy 1.26.4) raises the same ModuleNotFoundError.
Activity