Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instructor fails to work with OpenRouter when using Anthropic models #676

Open
2 tasks
syberkitten opened this issue May 17, 2024 · 0 comments
Open
2 tasks

Comments

@syberkitten
Copy link

syberkitten commented May 17, 2024

What Model are you using?

  • anthropic/cloaude-3-sonnet
  • gpt-4o

Describe the bug
Instructor fails to work work with openrouter when using Anthropic.

base_url = 'https://openrouter.ai/api/v1'
client = instructor.from_openai(AsyncOpenAI(base_url=base_url, api_key={openrouter_api_key}))
client.chat.completions.create(**params, model='anthropic/claude-3-sonnet', extra_headers=headers, response_model=response_model)

The we get is:
` ) -> BaseModel:
message = completion.choices[0].message
assert (

      len(message.tool_calls or []) == 1
    ), "Instructor does not support multiple tool calls, use List[Model] instead."

E AssertionError: Instructor does not support multiple tool calls, use List[Model] instead.`

If

Same code as above for 'openai/gpt-4o' works perfectly for OPENAI (using openrouter) and we get a response model as expected:
client.chat.completions.create(**params, model='openai/gpt-4o', extra_headers=headers, response_model=response_model)

If you will try to use the AsyncAnthropic instead of AsyncOpenAI the error gets even weirder and there is a long stack trace, this is the outmost error, something with not reaching the proper endpoint or something.

      raise self._make_status_error_from_response(err.response) from None

E anthropic.NotFoundError

Expected behavior
Allow to use Anthropic models over openrouter wether if it by using the openai client or anthropic (which would be better since the APIs are not identical)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant