-
Notifications
You must be signed in to change notification settings - Fork 904
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
text generation details not working when stream=False #1876
Comments
@uyeongkim I opened a similar issue at: huggingface/huggingface_hub#2281 Related issue for Since you use import requests
session = requests.Session()
# url = "http://0.0.0.0:80/generate_stream"
url = "http://0.0.0.0:80/generate"
data = {"inputs": "Today I am in Paris and", "parameters": {"max_new_tokens": 20}}
headers = {"Content-Type": "application/json"}
response = requests.post(url, json=data, headers=headers)
response = session.post(
url,
json=data,
headers=headers,
stream=False, # True,
)
# for line in response.iter_lines():
# print(f"line: `{line}`")
print(response.headers) |
It looks like this is a regression in huggingface_hub package, because it doesn't reproduce on older versions, like 0.20.0 |
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
System Info
I ran docker with model-id with downloaded lamma3 model, from huggingface.
And I requested with python code below
but It does not displays details,
TextGenerationOutput(generated_text='100% open-source and available on GitHub. It is distributed', details=None)
and server log is like
Information
Tasks
Reproduction
Expected behavior
text generate should give details instead of None
The text was updated successfully, but these errors were encountered: