[BUG] 简洁阐述问题 接入zhipu-api的在线大模型报错/ Concise description of the issue

[AIdrinkhotWater]

接入zhipu-api的在线大模型进行LLM对话报错，报错信息如下，请问是什么原因
Traceback (most recent call last):
File "/usr/local/serving/serving/server/utils.py", line 36, in wrap_done
await fn
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/base.py", line 385, in acall
raise e
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/base.py", line 379, in acall
await self._acall(inputs, run_manager=run_manager)
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/llm.py", line 275, in _acall
response = await self.agenerate([inputs], run_manager=run_manager)
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/llm.py", line 142, in agenerate
return await self.llm.agenerate_prompt(
File "/opt/conda/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 554, in agenerate_prompt
return await self.agenerate(
File "/opt/conda/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 514, in agenerate
raise exceptions[0]
File "/opt/conda/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 617, in _agenerate_with_cache
return await self._agenerate(
File "/opt/conda/lib/python3.10/site-packages/langchain_community/chat_models/openai.py", line 522, in _agenerate
return await agenerate_from_stream(stream_iter)
File "/opt/conda/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 87, in agenerate_from_stream
async for chunk in stream:
File "/opt/conda/lib/python3.10/site-packages/langchain_community/chat_models/openai.py", line 493, in _astream
if len(chunk["choices"]) == 0:
TypeError: object of type 'NoneType' has no len()

[dosubot]

The error you're encountering, "TypeError: object of type 'NoneType' has no len()", suggests that the code is attempting to measure the length of an object that is None. This issue arises in the line if len(chunk["choices"]) == 0:, indicating that chunk is expected to be a dictionary with a key "choices", but it's actually None.

To resolve this, ensure chunk is not None and contains the key "choices" before trying to access its length. You can modify the code to:

if chunk is not None and "choices" in chunk and len(chunk["choices"]) == 0:
    # Your logic here

This code snippet checks that chunk is not None and contains the "choices" key before evaluating its length, preventing the TypeError.

If chunk being None is unexpected, it might be due to the API not returning the expected data. Review the API documentation or the code handling these responses for more insight. Additionally, consider wrapping the code accessing chunk in a try-except block to log exceptions for more detailed insights into runtime errors:

try:
    if chunk is not None and "choices" in chunk and len(chunk["choices"]) == 0:
        # Your logic here
except Exception as e:
    logging.exception("Error while processing chunk: %s", e)

This approach will help identify why chunk is None and allow for more targeted troubleshooting.

Details

Sources

• chatglm3-6b模型llm模式，正常对话突然报错TypeError: object of type 'NoneType' has no len()
• 运行后在发送问题后出现如下报错
• [BUG] 简洁阐述问题 / Concise description of the issueTraceback (most recent call last): File "/data/soft/anaconda3/envs/langchain/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 535, in _run_script exec(code, module.dict) File "/data/chenhui/text_similarity_main/Langchain-Chatchat/webui.py", line 64, in pages[selected_page]["func"](api=api, is_lite=is_lite) File "/data/chenhui/text_similarity_main/Langchain-Chatchat/webui_pages/dialogue/dialogue.py", line 163, in dialogue_page running_models = list(api.list_running_models()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object is not iterable 报错的原因，给我正确的答案
• 对数据库 提问有问题，找到数据但不回答或数据库没有的什么都不回答
• ['Qwen-1_8B-Chat'] cpu]('Qwen-1_8B-Chat'] cpu #3497)
• ERROR: TypeError: Caught exception: object of type 'NoneType' has no len()
• tests/test_online_api.py
• server/knowledge_base/kb_summary/summary_chunk.py
• server/model_workers/zhipu.py
• knowledge_base/samples/content/test_files/langchain-ChatGLM_closed.jsonl
• server/chat/completion.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[QuanSir07]

同样，我也碰到这个，改了老半天，还是不能跑。