Skip to content

Error in training transformerLM #268

Closed

Description

@hwong39

Stage 0: Generate character level token_list from /workspace/model_scope/data/train2.txt
/opt/conda/bin/python /workspace/model_scope/FunASR/funasr/bin/tokenize_text.py --token_type char --input /workspace/model_scope/data/train2.txt --output ./data/exp/baseline_train_lm_transformer_zh_char_exp1/vocab.txt --non_linguistic_symbols none --field 2- --cleaner none --g2p none --write_vocabulary true --add_symbol ':0' --add_symbol ':1' --add_symbol ':2' --add_symbol ':-1'
2023-03-20 17:17:14,563 (tokenize_text:186) INFO: OOV rate = 0.0 %
stage 1: Data preparation
/opt/conda/bin/python /workspace/model_scope/FunASR/funasr/bin/aggregate_stats_dirs.py --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.1 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.2 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.3 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.4 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.5 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.6 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.7 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.8 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.9 --input_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/log/stats.10 --output_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1
stage 2: Training
run.sh: init method is file:///workspace/model_scope/FunASR/egs/aishell2/transformerLM/data/exp/baseline_train_lm_transformer_zh_char_exp1/ddp_init
Stage 3: Calc perplexity: /workspace/model_scope/data/test2.txt
/opt/conda/bin/python ../../../funasr/bin/lm_inference.py --output_dir ./data/exp/baseline_train_lm_transformer_zh_char_exp1/perplexity_test --ngpu 1 --batch_size 1 --train_config ./data/exp/baseline_train_lm_transformer_zh_char_exp1/config.yaml --model_file ./data/exp/baseline_train_lm_transformer_zh_char_exp1/valid.loss.ave.pth --data_path_and_name_and_type /workspace/model_scope/data/test2.txt,text,text --num_workers 1 --split_with_space false
Traceback (most recent call last):
File "../../../funasr/bin/lm_inference.py", line 406, in
main()
File "../../../funasr/bin/lm_inference.py", line 403, in main
inference(**kwargs)
File "../../../funasr/bin/lm_inference.py", line 68, in inference
**kwargs,
File "../../../funasr/bin/lm_inference.py", line 107, in inference_modelscope
train_config, model_file, device)
File "/workspace/model_scope/FunASR/funasr/tasks/abs_task.py", line 1918, in build_model_from_file
with config_file.open("r", encoding="utf-8") as f:
File "/opt/conda/lib/python3.7/pathlib.py", line 1208, in open
opener=self._opener)
File "/opt/conda/lib/python3.7/pathlib.py", line 1063, in _opener
return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'data/exp/baseline_train_lm_transformer_zh_char_exp1/config.yaml'

(modelscope image:ubuntu20.04-cuda11.3.0-py37-torch1.11.0-tf1.15.5-1.4.1)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions