TypeError: PeftConfig.__init__() got an unexpected keyword argument '_name_or_path' 报错怎么解决

[caijx168]

System Info

peft 版本为0.10.0

Who can help?

@pacman100 @younesbelkada @BenjaminBossan @sayakpaul @stevhliu

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder
• My own task or dataset (give details below)

Reproduction

部署langchain-chatchat 报错提示这个要怎么解决(Baichuan2) root@root1-System-Product-Name:/home/Baichuan2/Langchain-Chatchat# python startup.py -a

==============================Langchain-Chatchat Configuration==============================
操作系统：Linux-6.5.0-27-generic-x86_64-with-glibc2.35.
python版本：3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0]
项目版本：v0.2.10
langchain版本：0.0.354. fastchat版本：0.2.35

当前使用的分词器：ChineseRecursiveTextSplitter
当前启动的LLM模型：['peft-HitoGpt'] @ cuda
{'device': 'cuda',
'host': '0.0.0.0',
'infer_turbo': False,
'model_path': '/home/Baichuan2/Langchain-Chatchat/peft/peft-HitoGpt',
'model_path_exists': True,
'port': 20002}
当前Embbedings模型： bge-large-zh-v1.5 @ cuda
==============================Langchain-Chatchat Configuration==============================

2024-04-18 09:22:08,920 - startup.py[line:655] - INFO: 正在启动服务：
2024-04-18 09:22:08,920 - startup.py[line:656] - INFO: 如需查看 llm_api 日志，请前往 /home/Baichuan2/Langchain-Chatchat/logs
/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: 模型启动功能将于 Langchain-Chatchat 0.3.x重写,支持更多模式和加速启动，0.2.x中相关功能将废弃
warn_deprecated(
2024-04-18 09:22:11 | ERROR | stderr | INFO: Started server process [44838]
2024-04-18 09:22:11 | ERROR | stderr | INFO: Waiting for application startup.
2024-04-18 09:22:11 | ERROR | stderr | INFO: Application startup complete.
2024-04-18 09:22:11 | ERROR | stderr | INFO: Uvicorn running on http://0.0.0.0:20000 (Press CTRL+C to quit)
2024-04-18 09:22:11 | ERROR | stderr | Process model_worker - peft-HitoGpt:
2024-04-18 09:22:11 | ERROR | stderr | Traceback (most recent call last):
2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
2024-04-18 09:22:11 | ERROR | stderr | self.run()
2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/multiprocessing/process.py", line 108, in run
2024-04-18 09:22:11 | ERROR | stderr | self._target(*self._args, **self._kwargs)
2024-04-18 09:22:11 | ERROR | stderr | File "/home/Baichuan2/Langchain-Chatchat/startup.py", line 389, in run_model_worker
2024-04-18 09:22:11 | ERROR | stderr | app = create_model_worker_app(log_level=log_level, **kwargs)
2024-04-18 09:22:11 | ERROR | stderr | File "/home/Baichuan2/Langchain-Chatchat/startup.py", line 217, in create_model_worker_app
2024-04-18 09:22:11 | ERROR | stderr | worker = ModelWorker(
2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/fastchat/serve/model_worker.py", line 66, in init
2024-04-18 09:22:11 | ERROR | stderr | super().init(
2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/fastchat/serve/base_model_worker.py", line 47, in init
2024-04-18 09:22:11 | ERROR | stderr | self.conv = self.make_conv_template(conv_template, model_path)
2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/fastchat/serve/base_model_worker.py", line 75, in make_conv_template
2024-04-18 09:22:11 | ERROR | stderr | conv = get_conversation_template(model_path)
2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/fastchat/model/model_adapter.py", line 376, in get_conversation_template
2024-04-18 09:22:11 | ERROR | stderr | return adapter.get_default_conv_template(model_path)
2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/fastchat/model/model_adapter.py", line 654, in get_default_conv_template
2024-04-18 09:22:11 | ERROR | stderr | config = PeftConfig.from_pretrained(model_path)
2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/peft/config.py", line 151, in from_pretrained
2024-04-18 09:22:11 | ERROR | stderr | return cls.from_peft_type(**kwargs)
2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/peft/config.py", line 118, in from_peft_type
2024-04-18 09:22:11 | ERROR | stderr | return config_cls(**kwargs)
2024-04-18 09:22:11 | ERROR | stderr | TypeError: PeftConfig.init() got an unexpected keyword argument '_name_or_path'

Expected behavior

帮助我解决问题

[BenjaminBossan]

Sorry, I don't understand everything that you wrote.

There should be an adapter_config.json for your PEFT adapter, probably inside of /home/Baichuan2/Langchain-Chatchat/peft/peft-HitoGpt. Could you paste its content here?

[caijx168]

adapter_config.json 如下
cat adapter_config.json
{
"_name_or_path": "/home/ChatGLM/ChatGLM3/chatglm3-6b",
"add_bias_linear": false,
"add_qkv_bias": true,
"apply_query_key_layer_scaling": true,
"apply_residual_connection_post_layernorm": false,
"architectures": [
"ChatGLMForConditionalGeneration"
],
"attention_dropout": 0.0,
"attention_softmax_in_fp32": true,
"auto_map": {
"AutoConfig": "configuration_chatglm.ChatGLMConfig",
"AutoModel": "modeling_chatglm.ChatGLMForConditionalGeneration",
"AutoModelForCausalLM": "modeling_chatglm.ChatGLMForConditionalGeneration",
"AutoModelForSeq2SeqLM": "modeling_chatglm.ChatGLMForConditionalGeneration",
"AutoModelForSequenceClassification": "modeling_chatglm.ChatGLMForSequenceClassification"
},
"bias_dropout_fusion": true,
"classifier_dropout": null,
"eos_token_id": 2,
"ffn_hidden_size": 13696,
"fp32_residual_connection": false,
"hidden_dropout": 0.0,
"hidden_size": 4096,
"kv_channels": 128,
"layernorm_epsilon": 1e-05,
"model_type": "chatglm",
"multi_query_attention": true,
"multi_query_group_num": 2,
"num_attention_heads": 32,
"num_layers": 28,
"original_rope": true,
"pad_token_id": 0,
"padded_vocab_size": 65024,
"post_layer_norm": true,
"pre_seq_len": null,
"prefix_projection": false,
"quantization_bit": 0,
"rmsnorm": true,
"seq_length": 8192,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.38.2",
"use_cache": true,
"vocab_size": 65024
}

[BenjaminBossan]

Thanks. This does not look like a adapter_config.json from PEFT. Here is an example of a correct PEFT config file. The file you show looks like a config.json from transformers. Something must have gone wrong with the files when saving the model.

[caijx168]

我导出的模型文件是通过LLaMA-Factory微调之后合并导出的，目录结构如下

其原本生成的是config.json，然后我将其更名为adapter_config.json

[BenjaminBossan]

Why did you rename config.json to adapter_config.json? Those are not the same thing. Based on the file sizes you show, this is a full model, not a PEFT adapter. Try using AutoConfig or AutoModel from transformers to load the config/model.

If this was originally trained as a PEFT model, ensure that only the PEFT adapter is saved, not the whole model. For instance, don't merge the adapter before saving the model. I can't tell what exactly happened here though, you need to investigate that.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.