New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seems that setting logprobs=100 is not useful now. #14

Open

KL4805 opened this issue Sep 17, 2023 · 1 comment

KL4805 commented Sep 17, 2023

Hello authors,

I am really impressed with your efforts in creating this benchmark!

One small thing I notice is that OpenAI seems to limit the 'logprobs' argument to at most 5 (https://platform.openai.com/docs/api-reference/completions/create), while you set to 100. In this case, will your results be affected?

don-tpanic commented Sep 19, 2023

Wondering the same thing here. I'm guessing should have at least some effect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment