Repository navigation
Description
Stage 0: Generate character level token_list from /workspace/model_scope/data/train2.txt
/opt/conda/bin/python /workspace/model_scope/FunASR/funasr/bin/tokenize_text.py --token_type char --input /workspace/model_scope/data/train2.txt --output ./data/exp/baseline_train_lm_transformer_zh_char_exp1/vocab.txt --non_linguistic_symbols none --field 2- --cleaner none --g2p none --write_vocabulary true --add_symbol ':0' --add_symbol ':1' --add_symbol ':2' --add_symbol ':-1'
2023-03-20 17:17:14,563 (tokenize_text:186) INFO: OOV rate = 0.0 %
stage 1: Data preparation
/opt/conda/bin/python /workspace/model_scope/FunASR/funasr/bin/aggregate_stats_dirs.py --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.1 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.2 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.3 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.4 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.5 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.6 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.7 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.8 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.9 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.10 --output_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1
stage 2: Training
run.sh: init method is file:///workspace/model_scope/FunASR/egs/aishell2/transformerLM/data/exp/baseline_train_lm_transformer_zh_char_exp1/ddp_init
Stage 3: Calc perplexity: /workspace/model_scope/data/test2.txt
/opt/conda/bin/python ../../../funasr/bin/lm_inference.py --output_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/perplexity_test --ngpu 1 --batch_size 1 --train_config ./data/exp/baseline_train_lm_transformer_zh_char_exp1/config.yaml --model_file ./data/exp/baseline_train_lm_transformer_zh_char_exp1/valid.loss.ave.pth --data_path_and_name_and_type /workspace/model_scope/data/test2.txt,text,text --num_workers 1 --split_with_space false
Traceback (most recent call last):
File "../../../funasr/bin/lm_inference.py", line 406, in
main()
File "../../../funasr/bin/lm_inference.py", line 403, in main
inference(**kwargs)
File "../../../funasr/bin/lm_inference.py", line 68, in inference
**kwargs,
File "../../../funasr/bin/lm_inference.py", line 107, in inference_modelscope
train_config, model_file, device)
File "/workspace/model_scope/FunASR/funasr/tasks/abs_task.py", line 1918, in build_model_from_file
with config_file.open("r", encoding="utf-8") as f:
File "/opt/conda/lib/python3.7/pathlib.py", line 1208, in open
opener=self._opener)
File "/opt/conda/lib/python3.7/pathlib.py", line 1063, in _opener
return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'data/exp/baseline_train_lm_transformer_zh_char_exp1/config.yaml'
(modelscope image:ubuntu20.04-cuda11.3.0-py37-torch1.11.0-tf1.15.5-1.4.1)
Activity