Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors for some reason. #62

Open
MistakingManx opened this issue Apr 24, 2024 · 3 comments
Open

Errors for some reason. #62

MistakingManx opened this issue Apr 24, 2024 · 3 comments

Comments

@MistakingManx
Copy link

Traceback (most recent call last):
  File "D:\anaconda3\envs\TTS\lib\site-packages\trainer\trainer.py", line 1833, in fit
    self._fit()
  File "D:\anaconda3\envs\TTS\lib\site-packages\trainer\trainer.py", line 1785, in _fit
    self.train_epoch()
  File "D:\anaconda3\envs\TTS\lib\site-packages\trainer\trainer.py", line 1503, in train_epoch
    for cur_step, batch in enumerate(self.train_loader):
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\utils\data\dataloader.py", line 631, in __next__
    data = self._next_data()
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\utils\data\dataloader.py", line 1346, in _next_data
    return self._process_data(data)
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\utils\data\dataloader.py", line 1372, in _process_data
    data.reraise()
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\_utils.py", line 722, in reraise
    raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\utils\data\_utils\worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\utils\data\_utils\fetch.py", line 54, in fetch
    return self.collate_fn(data)
  File "D:\AI\Testing\TTS\TTS\tts\models\vits.py", line 343, in collate_fn
    wav_padded[i, :, : wav.size(1)] = torch.FloatTensor(wav)
RuntimeError: The expanded size of the tensor (1) must match the existing size (2) at non-singleton dimension 0.  Target sizes: [1, 44982].  Tensor sizes: [2, 44982]
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\AI\Testing\TTS\train_vits.py", line 91, in <module>
    main()
  File "D:\AI\Testing\TTS\train_vits.py", line 85, in main
    trainer.fit()
  File "D:\anaconda3\envs\TTS\lib\site-packages\trainer\trainer.py", line 1860, in fit
    remove_experiment_folder(self.output_path)
  File "D:\anaconda3\envs\TTS\lib\site-packages\trainer\generic_utils.py", line 77, in remove_experiment_folder
    fs.rm(experiment_path, recursive=True)
  File "D:\anaconda3\envs\TTS\lib\site-packages\fsspec\implementations\local.py", line 185, in rm
    shutil.rmtree(p)
  File "D:\anaconda3\envs\TTS\lib\shutil.py", line 750, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "D:\anaconda3\envs\TTS\lib\shutil.py", line 620, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "D:\anaconda3\envs\TTS\lib\shutil.py", line 618, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'D:/AI/Testing/TTS/vits_premier1-April-23-2024_09+49PM-dbf1a08a\\trainer_0_log.txt'

I lack sufficient Python understanding to solve this issue myself.

@MistakingManx
Copy link
Author

Oh right, the file. train_vits_win.py.

@thorstenMueller
Copy link
Owner

I've run into the same problem with "_The process cannot access the file because it is being used by another process" while doing some tests on Windows, but i'm not sure how i solved it. But this happens when training already crashed and the process tries cleaning up the training folder.

  • Did you run your command line with admin privilieges?
  • Is there any file open from the training directory in explorer?

@MistakingManx
Copy link
Author

MistakingManx commented Apr 24, 2024

The issue I'm having is primarily the RuntimeError: The expanded size of the tensor (1) must match the existing size (2) at non-singleton dimension 0. Target sizes: [1, 44982]. Tensor sizes: [2, 44982] issue. I don't understand what this is.

(TTS) D:\AI\Testing\TTS>python train_vits.py
 > Setting up Audio Processor...
 | > sample_rate:44100
 | > resample:False
 | > num_mels:80
 | > log_func:np.log10
 | > min_level_db:0
 | > frame_shift_ms:None
 | > frame_length_ms:None
 | > ref_level_db:None
 | > fft_size:1024
 | > power:None
 | > preemphasis:0.0
 | > griffin_lim_iters:None
 | > signal_norm:None
 | > symmetric_norm:None
 | > mel_fmin:0
 | > mel_fmax:None
 | > pitch_fmin:None
 | > pitch_fmax:None
 | > spec_gain:20.0
 | > stft_pad_mode:reflect
 | > max_norm:1.0
 | > clip_norm:True
 | > do_trim_silence:False
 | > trim_db:60
 | > do_sound_norm:False
 | > do_amp_to_db_linear:True
 | > do_amp_to_db_mel:True
 | > do_rms_norm:False
 | > db_level:None
 | > stats_path:None
 | > base:10
 | > hop_length:256
 | > win_length:1024
 | > Found 15425 files in D:\AI\Testing\TTS\dataset\dataset\LJSpeech-1.1
 > Training Environment:
 | > Backend: Torch
 | > Mixed precision: False
 | > Precision: float32
 | > Current device: 0
 | > Num. of GPUs: 1
 | > Num. of CPUs: 12
 | > Num. of Torch Threads: 6
 | > Torch seed: 54321
 | > Torch CUDNN: True
 | > Torch CUDNN deterministic: False
 | > Torch CUDNN benchmark: False
 | > Torch TF32 MatMul: False
 > Start Tensorboard: tensorboard --logdir=D:\AI\Testing\TTS\vits_premier1-April-24-2024_07+11PM-dbf1a08a

 > Model has 83059180 parameters

 > EPOCH: 0/1000
 --> D:\AI\Testing\TTS\vits_premier1-April-24-2024_07+11PM-dbf1a08a


> DataLoader initialization
| > Tokenizer:
        | > add_blank: True
        | > use_eos_bos: False
        | > use_phonemes: True
        | > phonemizer:
                | > phoneme language: en-us
                | > phoneme backend: espeak
| > Number of instances : 15271
 | > Preprocessing samples
 | > Max text length: 200
 | > Min text length: 2
 | > Avg text length: 82.88874336978587
 |
 | > Max audio length: 668115
 | > Min audio length: 42336
 | > Avg audio length: 217587.1818479471
 | > Num. instances discarded samples: 0
 | > Batch group size: 20.

 > TRAINING (2024-04-24 19:13:15)
Traceback (most recent call last):
  File "D:\anaconda3\envs\TTS\lib\site-packages\trainer\trainer.py", line 1833, in fit
    self._fit()
  File "D:\anaconda3\envs\TTS\lib\site-packages\trainer\trainer.py", line 1785, in _fit
    self.train_epoch()
  File "D:\anaconda3\envs\TTS\lib\site-packages\trainer\trainer.py", line 1503, in train_epoch
    for cur_step, batch in enumerate(self.train_loader):
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\utils\data\dataloader.py", line 631, in __next__
    data = self._next_data()
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\utils\data\dataloader.py", line 1346, in _next_data
    return self._process_data(data)
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\utils\data\dataloader.py", line 1372, in _process_data
    data.reraise()
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\_utils.py", line 722, in reraise
    raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\utils\data\_utils\worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "D:\anaconda3\envs\TTS\lib\site-packages\torch\utils\data\_utils\fetch.py", line 54, in fetch
    return self.collate_fn(data)
  File "D:\AI\Testing\TTS\TTS\tts\models\vits.py", line 343, in collate_fn
    wav_padded[i, :, : wav.size(1)] = torch.FloatTensor(wav)
RuntimeError: The expanded size of the tensor (1) must match the existing size (2) at non-singleton dimension 0.  Target sizes: [1, 44982].  Tensor sizes: [2, 44982]

Running with administrator privileges results in the same error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants