-
Notifications
You must be signed in to change notification settings - Fork 327
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ChatGLM2 lora finetuning 加载 lora 参数:RuntimeError: Expected 4-dimensional input for 4-dimensional weight [3072, 32, 1, 1], but got 3-dimensional input of size [1, 64, 4096] instead #150
Labels
Comments
用这种方式加载,也是一样的问题: model = AutoModel.from_pretrained(model_name_or_path, trust_remote_code=True, device_map='auto', torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(model, peft_model_id)
model = model.eval() |
如果这样加载(先做一次 eval): tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name_or_path, trust_remote_code=True, device_map='auto', torch_dtype=torch.bfloat16)
model = model.eval()
model = PeftModel.from_pretrained(model, peft_model_id) 在这一步会出现如下问题: ValueError: We need an `offload_dir` to dispatch this model according to this `device_map`, the following submodules need to be offloaded: base_model.model.transformer.encoder.layers.1,
base_model.model.transformer.encoder.layers.2, base_model.model.transformer.encoder.layers.3, base_model.model.transformer.encoder.layers.4
错误位置:
/home/beeservice/.conda/envs/pt/lib/python3.10/site-packages/peft/peft_model.py:177 in │
│ from_pretrained │
│ │
│ 174 │ │ │ │ device_map = infer_auto_device_map( │
│ 175 │ │ │ │ │ model, max_memory=max_memory, no_split_module_classes=no_split_modul │
│ 176 │ │ │ │ ) │
│ ❱ 177 │ │ │ model = dispatch_model(model, device_map=device_map) │
│ 178 │ │ │ hook = AlignDevicesHook(io_same_device=True) │
│ 179 │ │ │ if model.peft_config.peft_type == PeftType.LORA: │
│ 180 │ │ │ │ add_hook_to_module(model.base_model.model, hook) │ |
不知道你是不是用我的代码训练的。也有可能是transformers和peft包的版本问题。建议更新一下试一试。 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
加载 lora 后的 model 信息:
加载方法:
出现错误,错误行号:
The text was updated successfully, but these errors were encountered: