New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generation issues with inputs which contain commas, punctuation, and ellipsis #130
Comments
Are you able to share some of the texts? It tends to be dependent on the speaker reference used, and it will be improved in one of the 2 upcoming releases |
This issue has seemingly been resolved by finetuning the base model for 20 epochs on hand-crafted voice data which includes commas, multiple sentences, and ellipses (41 < len < 261, n=600, 0.2 train split, no LR decay, default training config). Still seems to have trouble having correct / accurate prosody -- even when zero-shot cloning voice is in the 'train' segmentation of the dataset. We call this effect the "William Shatner" effect... 😸 |
Thanks for the update @somewheresy & sorry for the delay in my response - I'll go ahead & close out the issue but feel free to reopen if there's anything else to raise. |
When inputs contain commas and periods, the output generations often create too much interstitial space / silence between words before and after the characters.
When ellipsis are present in the input, instead of creating more interstitial space (silence), the model tends to hallucinate.
Could this be from removal of characters in the 100K hour training set?
The text was updated successfully, but these errors were encountered: