使用中文的数据进行重新训练，模型还能具有处理英文的能力吗?处理英文的效果如何呢？ #860
Nanamumuhan
started this conversation in
General
Replies: 1 comment 4 replies
-
|
不清楚你说的“重新训练”是什么意思，随机初始化？在原版llama上继续训练？ |
Beta Was this translation helpful? Give feedback.
4 replies
-
|
就是在现有的英语预训练模型上fine tuning一个新的中文的模型，模型参数在训练过程中会发生改变。 |
Beta Was this translation helpful? Give feedback.
-
|
后者是使用英文预训练模型初始化中文的预训练模型后在训练中文的模型，这样的花中文的预训练模型还能保留原有的英文的能力吗？ |
Beta Was this translation helpful? Give feedback.
-
|
你只训中文那么英文能力肯定是要丢失一些的，如果不想出现灾难性遗忘问题可以尝试同时使用中英文数据。 |
Beta Was this translation helpful? Give feedback.
-
|
那是不是得使用中英文对齐数据 |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
使用中文的数据进行重新训练，模型还能具有处理英文的能力吗?处理英文的效果如何呢？
Beta Was this translation helpful? Give feedback.
All reactions