Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to train on a new dataset after the engine had been created? #622

Open
AlyShmahell opened this issue Feb 12, 2023 · 0 comments
Open

Comments

@AlyShmahell
Copy link

AlyShmahell commented Feb 12, 2023

I have an engine I created with ./mmt create using an old dataset I had, and I would like to retrain it again using a new dataset for more fine-tuning.
I've tried ./mmt train --from-model but it messes up the BLEU Score.
I also tried creating a new engine from scratch with ./mmt create to train it on both datasets together, but I got an exit code -11 while training, i'm assuming the jvm ran out of memory at some point (it was using a lot while I was monitoring memory usage).
I've seen from other issues people suggesting ./mmt memory import -p parallel_src parallel_tgt -e model_name, but i'm not sure how much this impacts the performance or translation speed.
Any tips or suggestions are appreciated, thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant