convert_to_openai_function not generating properly with nested BaseModels

[eyurtsev]

Discussed in #17481

^{Originally posted by mimichelow February 13, 2024}

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_core.utils.function_calling import convert_to_openai_function
from enum import Enum
from pydantic import BaseModel, Field
from typing import List

class Tool(Enum):
    WEB_SEARCH = "web-search"
    RSS_FEED_SCRAPER = "rss-feed-scraper"
    USER_INPUT = "user-input"

class Task(BaseModel):
    """Tasks based on the objective"""
    id: int = Field(description="Create an ID and make sure all task IDs are in chronological order")
    task: str = Field(description="Task description should be detailed. TASK Example: 'Look up AI news from today (May 27, 2023) and write a poem'")
    tool: Tool = Field(description="Current tool options are [text-completion] [web-search] [rss-feed-scraper]")

class TaskList(BaseModel):
    """List of tasks"""
    task_list: List[Task] = Field(description="List of tasks")

extraction_functions = [convert_to_openai_function(TaskList)]

Description

I'm trying to migrate from the deprecated convert_pydantic_to_openai_function with the previous Pydantic class, but the new function is not going beyond the first level. The produced output is

[
  {
    "name": "TaskList",
    "description": "List of tasks",
    "parameters": {
      "type": "object",
      "properties": {},
      "required": ["task_list"]
    }
  }
]

Is this a known bug, do i have to manually produce the full schema to pass as a dictionary now? I can't find anything in how to generate the full openai function using the new function.

Thanks

System Info

System Information

OS: Windows
OS Version: 10.0.22621
Python Version: 3.11.6 (tags/v3.11.6:8b6ee5b, Oct 2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)]
Package Information

---

langchain_core: 0.1.22
langchain: 0.1.6
langchain_community: 0.0.19
langsmith: 0.0.87
langchain_openai: 0.0.5
Packages not installed (Not Necessarily a Problem)

---

The following packages were not found:

langgraph
langserve

[conditionedstimulus]

Hi @eyurtsev, @mimichelow!

Upon investigating the error, it appears that the issue lies with the import of pydantic. The nested function seems to only work correctly when imported as from langchain_core.pydantic_v1 import BaseModel, Field instead of from pydantic import Field, BaseModel.

Within the convert_to_openai_function, when dealing with a standard Pydantic BaseModel, it's reaching the elif callable(function) and it calls the convert_python_function_to_openai_function(function) method part instead of the section where convert_pydantic_to_openai_function (which handles nested versions) is called.

The chunk in the code: (from line 281 langchain_core/utils/function_calling.py)

if isinstance(function, dict):
        return function
elif isinstance(function, type) and issubclass(function, BaseModel):
        return cast(Dict, convert_pydantic_to_openai_function(function))
elif isinstance(function, BaseTool):
        return format_tool_to_openai_function(function)
elif callable(function):
        return convert_python_function_to_openai_function(function)

Modified script:

from langchain_core.utils.function_calling import convert_to_openai_function
from enum import Enum
from langchain_core.pydantic_v1 import BaseModel, Field
from typing import List

class Tool(Enum):
    WEB_SEARCH = "web-search"
    RSS_FEED_SCRAPER = "rss-feed-scraper"
    USER_INPUT = "user-input"

class Task(BaseModel):
    """Tasks based on the objective"""
    id: int = Field(description="Create an ID and make sure all task IDs are in chronological order")
    task: str = Field(description="Task description should be detailed. TASK Example: 'Look up AI news from today (May 27, 2023) and write a poem'")
    tool: Tool = Field(description="Current tool options are [text-completion] [web-search] [rss-feed-scraper]")

class TaskList(BaseModel):
    """List of tasks"""
    task_list: List[Task] = Field(description="List of tasks")

extraction_functions = [convert_to_openai_function(TaskList)]

extraction_functions

Output:

[{'name': 'TaskList',
  'description': 'List of tasks',
  'parameters': {'type': 'object',
   'properties': {'task_list': {'description': 'List of tasks',
     'type': 'array',
     'items': {'description': 'Tasks based on the objective',
      'type': 'object',
      'properties': {'id': {'description': 'Create an ID and make sure all task IDs are in chronological order',
        'type': 'integer'},
       'task': {'description': "Task description should be detailed. TASK Example: 'Look up AI news from today (May 27, 2023) and write a poem'",
        'type': 'string'},
       'tool': {'description': 'Current tool options are [text-completion] [web-search] [rss-feed-scraper]',
        'allOf': [{'title': 'Tool',
          'description': 'An enumeration.',
          'enum': ['web-search', 'rss-feed-scraper', 'user-input']}]}},
      'required': ['id', 'task', 'tool']}}},
   'required': ['task_list']}}]

[hinthornw]

Ya it also silently will drop some special keys - looking forward to upgrading to pydantic v2 in the future

[meirm]

Updating the notebooks from the course on "Functions, Tools and Agents with LangChain" at DeepLearning.ai by Harrison Chase has similar issue.
class Information(BaseModel):
"""Information to extract."""
people: List[Person] = Field(description="List of info about people")

convert_to_openai_function(Information)

{'name': 'Information',
'description': 'Information to extract.',
'parameters': {'type': 'object', 'properties': {}, 'required': ['people']}}

from langchain.utils.openai_functions import convert_pydantic_to_openai_function
convert_pydantic_to_openai_function(Information)
{'name': 'Information',
'description': 'Information to extract.',
'parameters': {'$defs': {'Person': {'description': 'Information about a person.',
'properties': {'name': {'description': "person's name", 'type': 'string'},
'age': {'anyOf': [{'type': 'integer'}, {'type': 'null'}],
'description': "person's age"}},
'required': ['name', 'age'],
'type': 'object'}},
'properties': {'people': {'description': 'List of info about people',
'items': {'description': 'Information about a person.',
'properties': {'name': {'description': "person's name", 'type': 'string'},
'age': {'anyOf': [{'type': 'integer'}, {'type': 'null'}],
'description': "person's age"}},
'required': ['name', 'age'],
'type': 'object'},
'type': 'array'}},
'required': ['people'],
'type': 'object'}}

Using:
from langchain_core.pydantic_v1 import BaseModel, Field

solves the problem.