Welcome to german-tts Discussions!

[monatis]

👋 Welcome!

I'm using Discussions as a place to connect with people finding this repo useful. We hope that you:

• Tell us what are you using it for.
• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this repo.

[eqikkwkp25-cyber]

Hello @monatis,

i do not think that your repository resp. the provided models are actively used as the training for tacotron2 and mb-melgan are still ongoing, right?

If everything goes well I personally hope to use it for a setup with a RPI + Matrix Creator + Kalliope + Kaldi STT. Something contemporary to replace pico2wave as a TTS backend is appreciated:-) Besides Thorstens dataset there are only few others for the German language and a lack of available pretrained models.

Thank you for your efforts.

[monatis]

Hi @eqikkwkp25-cyber 👋 Thanks for your message 🙏
In fact training is complete but I am continuing to evaluate models checkpointed at different steps to find out the best one in quality of synthesis. As it may be overfitted a little bit towards the end of training, a checkpoint at an earlier step may work better. I'll publish a post very soon to summarize the whole process of training and such evaluation results 🙂
After deciding the best checkpoint for Tacotron2, I'll train a FastSpeech2 model with that, and it can do a better and performant job with your RPi setup --Tacotron2 may be a little slower there.

[eqikkwkp25-cyber]

Fastspeech2 might be a good idea @monatis. I already tried it out with pre-trained English models, in combination with Hifi-GAN vocoder if i remember correctly. BTW, currently i use a local Intel CPU based server for STT and TTS purposes but i might re-evaluate for instance the TF lite model you provide.

On the other hand, i am also interested in the German Karlsson dataset, some details can be found here https://discourse.mozilla.org/t/german-karlsson-models-and-notebook/71458

[monatis]

German is not my native language, but M-ailabs dataset, which also includes Karlsson, is said to have some problems in annotations (mismatches between the text and audio, cut-off syllables in the beginning or at the end, etc.). Also, I believe that audio books are not well suited for TTS training because these recordings usually contain highly emotional and theatric speech to give specific voices to characters in the book (and that's why they sound more natural to us but this theatric speech is undesirable for a TTS dataset). Therefore, I believe that the whole M-ailabs dataset is more suited for STT than TTS. And as a next step, it is one of my future plans to train a state-of-the-art STT model in German with Common Voice and M-ailabs combined (I'm just looking for obtaining some free cloud credits for this project).

I really would love to hear about the results of your benchmarking. Some TF Lite models may be slower on Intel CPUs and it is likely that this is the case with with Tacotron2 in particular. Anyway it'll be great to if you can share your benchmarking with me 🎉

[eqikkwkp25-cyber]

Thank you for your explanations regarding the usage of datasets based on audio books for TTS (and STT). I agree on most of your comments but personally i think that the advantages outweigh the disadvantages and it is highly dependent on the speaker, the content spoken etc. Different ears, different preferences :-)

It might be a good idea to add some realtime factor computations RTF to the inference notebook(s) for benchmark and comparison purposes.

[eqikkwkp25-cyber]

Just to give you an initial idea: Inference is about 0.5 RTF using Tacatron2 and MB-Melgan here on an older core i5 CPU computer .

[danielegobbetti]

Hello @monatis , sorry if I put some pressure, but I would like to know if you plan releasing a new release (with fastspeech2 as announced) soon or if the plans have changed.

Alternatively some directions were welcome: my aim is to plug this tts into android by using https://github.com/TensorSpeech/TensorFlowTTS , I was able to integrate their android demo with the system's TTS engine, but I would like to add a German voice to it. I've seen this dataset is supported and I was able to execute the colab that's mentioned, but it only builds a Tacotron2 tflite file and my knowledge in this field is too limited to try "converting" this to fastspeech2 (assuming that it's at all possible).

Thanks in advance!