-
Notifications
You must be signed in to change notification settings - Fork 274
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
how to use stage2 ckpt fine-tuning stage3? #102
Comments
Hi, please refer to the issue #49 for the continuous fine-tuning. And all our models are fully finetunded. Actually, I did not try LoRA. This could be checked and supported soon. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
First, I modified the scripts/llama/train/stage_1_2_full_v7b_336_hr_768.sh, changing parameter ”--model_name_or_path” to the stage2 checkpoint “MGM-7B”, and then got loar model_path.
Afterwards, using scripts/merge_lora_weights.py to merge the base and lora. But I found that it does not work. Compared to LLava, the mgm.model.builder.load_pretrained_model method seems to be lacking several components, the method don't load peftmodel.
The text was updated successfully, but these errors were encountered: