Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

模型不支持 Model not supported, name: fuzimingcha_v1.0, format: pytorch, size: 6, quantization: 8-bit #1430

Open
zjx140 opened this issue May 6, 2024 · 2 comments
Open

模型不支持 Model not supported, name: fuzimingcha_v1.0, format: pytorch, size: 6, quantization: 8-bit #1430

zjx140 opened this issue May 6, 2024 · 2 comments
Labels
question Further information is requested
Milestone
v0.11.3

Comments

@zjx140
Copy link

zjx140 commented May 6, 2024

使用基于ChatGLM-6B的夫子明察法律大模型，部署出现问题。
xinference=0.10.3

报错如下，使用任何量化方式都有该问题

2024-05-06 19:58:13,589 xinference.model.llm.llm_family 660 INFO     Caching from URI: /mnt/d/fuzi-mingcha-v1_0
2024-05-06 19:58:13,593 xinference.model.llm.llm_family 660 INFO     Cache /mnt/d/fuzi-mingcha-v1_0 exists
2024-05-06 19:58:13,593 xinference.core.worker 660 ERROR    Failed to load model fuzimingcha_v1.0-1-0
Traceback (most recent call last):
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/worker.py", line 673, in launch_builtin_model
    model, model_description = await asyncio.to_thread(
  File "/root/anaconda3/envs/xinference/lib/python3.9/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/root/anaconda3/envs/xinference/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/model/core.py", line 67, in create_model_instance
    return create_llm_model_instance(
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/model/llm/core.py", line 212, in create_llm_model_instance
    raise ValueError(
ValueError: Model not supported, name: fuzimingcha_v1.0, format: pytorch, size: 6, quantization: 8-bit
2024-05-06 19:58:13,617 xinference.api.restful_api 596 ERROR    [address=0.0.0.0:23614, pid=660] Model not supported, name: fuzimingcha_v1.0, format: pytorch, size: 6, quantization: 8-bit
Traceback (most recent call last):
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/api/restful_api.py", line 731, in launch_model
    model_uid = await (await self._get_supervisor_ref()).launch_builtin_model(
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/context.py", line 227, in send
    return self._process_result_message(result)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/pool.py", line 659, in send
    result = await self._run_coro(message.message_id, coro)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
    return await coro
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/api.py", line 384, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
    result = await result
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/supervisor.py", line 850, in launch_builtin_model
    await _launch_model()
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/supervisor.py", line 814, in _launch_model
    await _launch_one_model(rep_model_uid)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/supervisor.py", line 796, in _launch_one_model
    await worker_ref.launch_builtin_model(
  File "xoscar/core.pyx", line 284, in __pyx_actor_method_wrapper
    async with lock:
  File "xoscar/core.pyx", line 287, in xoscar.core.__pyx_actor_method_wrapper
    result = await result
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/utils.py", line 45, in wrapped
    ret = await func(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/worker.py", line 673, in launch_builtin_model
    model, model_description = await asyncio.to_thread(
  File "/root/anaconda3/envs/xinference/lib/python3.9/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/root/anaconda3/envs/xinference/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/model/core.py", line 67, in create_model_instance
    return create_llm_model_instance(
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/model/llm/core.py", line 212, in create_llm_model_instance
    raise ValueError(
ValueError: [address=0.0.0.0:23614, pid=660] Model not supported, name: fuzimingcha_v1.0, format: pytorch, size: 6, quantization: 8-bit
@zjx140 zjx140 added the question Further information is requested label May 6, 2024
@XprobeBot XprobeBot added this to the v0.11.0 milestone May 6, 2024
@qinxuye
Copy link
Contributor

qinxuye commented May 6, 2024

注册自定义模型了吗？

@zjx140
Copy link
Author

zjx140 commented May 6, 2024 •
edited

使用过注册自定义模型的办法，但是报了另外一个错误。在[https://github.com/irlab-sdu/fuzi.mingcha/issues/13]中给出了使用AutoModel类的解决方法，但是xinference源代码里面是使用的是AutoModelForCausalLM类，所以出现了这个错误。

Traceback (most recent call last):
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/worker.py", line 697, in launch_builtin_model
    await model_ref.load()
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/context.py", line 227, in send
    return self._process_result_message(result)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/pool.py", line 659, in send
    result = await self._run_coro(message.message_id, coro)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
    return await coro
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/api.py", line 384, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 524, in xoscar.core._BaseActor.__on_receive__
    result = func(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/model.py", line 239, in load
    self._model.load()
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/model/llm/pytorch/core.py", line 243, in load
    self._model, self._tokenizer = self._load_model(**kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/model/llm/pytorch/core.py", line 130, in _load_model
    model = AutoModelForCausalLM.from_pretrained(
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    raise ValueError(
ValueError: [address=0.0.0.0:59593, pid=1450] Unrecognized configuration class <class 'transformers_modules.fuzi-mingcha-v1_0.configuration_chatglm.ChatGLMConfig'> for this kind of AutoModel: AutoModelForCausalLM.
Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, LlamaConfig, CodeGenConfig, CohereConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DbrxConfig, ElectraConfig, ErnieConfig, FalconConfig, FuyuConfig, GemmaConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, JambaConfig, LlamaConfig, MambaConfig, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MptConfig, MusicgenConfig, MusicgenMelodyConfig, MvpConfig, OlmoConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PersimmonConfig, PhiConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, RecurrentGemmaConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, Speech2Text2Config, StableLmConfig, Starcoder2Config, TransfoXLConfig, TrOCRConfig, WhisperConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, OmniLMMConfig.
2024-05-06 18:57:08,028 xinference.api.restful_api 596 ERROR    [address=0.0.0.0:59593, pid=1450] Unrecognized configuration class <class 'transformers_modules.fuzi-mingcha-v1_0.configuration_chatglm.ChatGLMConfig'> for this kind of AutoModel: AutoModelForCausalLM.
Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, LlamaConfig, CodeGenConfig, CohereConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DbrxConfig, ElectraConfig, ErnieConfig, FalconConfig, FuyuConfig, GemmaConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, JambaConfig, LlamaConfig, MambaConfig, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MptConfig, MusicgenConfig, MusicgenMelodyConfig, MvpConfig, OlmoConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PersimmonConfig, PhiConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, RecurrentGemmaConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, Speech2Text2Config, StableLmConfig, Starcoder2Config, TransfoXLConfig, TrOCRConfig, WhisperConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, OmniLMMConfig.
Traceback (most recent call last):
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/api/restful_api.py", line 731, in launch_model
    model_uid = await (await self._get_supervisor_ref()).launch_builtin_model(
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/context.py", line 227, in send
    return self._process_result_message(result)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/pool.py", line 659, in send
    result = await self._run_coro(message.message_id, coro)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
    return await coro
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/api.py", line 384, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
    result = await result
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/supervisor.py", line 850, in launch_builtin_model
    await _launch_model()
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/supervisor.py", line 814, in _launch_model
    await _launch_one_model(rep_model_uid)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/supervisor.py", line 796, in _launch_one_model
    await worker_ref.launch_builtin_model(
  File "xoscar/core.pyx", line 284, in __pyx_actor_method_wrapper
    async with lock:
  File "xoscar/core.pyx", line 287, in xoscar.core.__pyx_actor_method_wrapper
    result = await result
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/utils.py", line 45, in wrapped
    ret = await func(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/worker.py", line 697, in launch_builtin_model
    await model_ref.load()
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/context.py", line 227, in send
    return self._process_result_message(result)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/pool.py", line 659, in send
    result = await self._run_coro(message.message_id, coro)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
    return await coro
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xoscar/api.py", line 384, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 524, in xoscar.core._BaseActor.__on_receive__
    result = func(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/core/model.py", line 239, in load
    self._model.load()
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/model/llm/pytorch/core.py", line 243, in load
    self._model, self._tokenizer = self._load_model(**kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/model/llm/pytorch/core.py", line 130, in _load_model
    model = AutoModelForCausalLM.from_pretrained(
  File "/root/anaconda3/envs/xinference/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    raise ValueError(
ValueError: [address=0.0.0.0:59593, pid=1450] Unrecognized configuration class <class 'transformers_modules.fuzi-mingcha-v1_0.configuration_chatglm.ChatGLMConfig'> for this kind of AutoModel: AutoModelForCausalLM.
Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, LlamaConfig, CodeGenConfig, CohereConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DbrxConfig, ElectraConfig, ErnieConfig, FalconConfig, FuyuConfig, GemmaConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, JambaConfig, LlamaConfig, MambaConfig, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MistralConfig, MixtralConfig, MptConfig, MusicgenConfig, MusicgenMelodyConfig, MvpConfig, OlmoConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PersimmonConfig, PhiConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, RecurrentGemmaConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, Speech2Text2Config, StableLmConfig, Starcoder2Config, TransfoXLConfig, TrOCRConfig, WhisperConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, OmniLMMConfig.

@XprobeBot XprobeBot modified the milestones: v0.11.0, v0.11.1, v0.11.2 May 11, 2024
@XprobeBot XprobeBot modified the milestones: v0.11.2, v0.11.3 May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
v0.11.3
Development

No branches or pull requests
3 participants
@qinxuye @zjx140 @XprobeBot