Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nan loss during training #5

Open
dominicmini opened this issue Aug 8, 2022 · 7 comments
Open

nan loss during training #5

dominicmini opened this issue Aug 8, 2022 · 7 comments

Comments

@dominicmini
Copy link

hi you, I'm trying to train model AdaSpeech from your project. However when I train pretrained model I get nan loss during training.
do you have any way to fix it from your code?
image

@tuanh123789
Copy link
Owner

Yes, i'll update the newest code version to fix it soon. Because the new version of MFA tool make the preprocess code generate 0 frame for some phoneme.

@tuanh123789
Copy link
Owner

tuanh123789 commented Aug 8, 2022

in preprocessor/preprocess.py replace function average_mel_by_duration by this code:
image

@dominicmini
Copy link
Author

Thanks a lot, I fixed it!
Do you have a pretrained model training on English or other language?

@manhph2211
Copy link

@dominicmini I met the same issue but the above solution does not solve the problem 😢 Do you have any advice for me ❤️

@manhph2211
Copy link

@tuanh123789 Hi bro:v Can you tell me about the effectiveness of the average_mel_by_duration function 💯

@HandsLing
Copy link

@tuanh123789 Hi bro:v Can you tell me about the effectiveness of the average_mel_by_duration function 💯

i met the same issue, do you solve this question?

@tuntun990606
Copy link

@tuanh123789 Hi bro:v Can you tell me about the effectiveness of the average_mel_by_duration function 💯

i met the same issue, do you solve this question?

Hello, I encountered such a problem at the beginning, and now it has been solved. At the beginning, I made changes completely according to the pictures replied by the author. The preprocess.py 338 line still reported an error. Later, I deleted the last comma of the author, so preprocess.py would not report an error. Then you can train normally

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants