Validated Schema

[jxnl]

Is your feature request related to a problem? Please describe.

I'm the maintainer of Instructor which uses pydantic (python's validation) library to improve the sdk usaibility. If I wanted to do something like this in ruby, do you recommend ruby-openai support it? or should there be some seperate library that patchs this one, and adds the various new keywords?

In python it looks like

import instructor
from openai import OpenAI
from pydantic import BaseModel

# This enables response_model keyword
# from client.chat.completions.create
client = instructor.patch(OpenAI())

class UserDetail(BaseModel):
    name: str
    age: int

user = client.chat.completions.create(
    model="gpt-3.5-turbo",
    response_model=UserDetail,
    messages=[
        {"role": "user", "content": "Extract Jason is 25 years old"},
    ]
)

assert isinstance(user, UserDetail)
assert user.name == "Jason"
assert user.age == 25

In ruby you can imagine using

UserSchema = Dry::Validation.Schema do
  required(:name).filled(:str?)
  required(:age).filled(:int?, gt?: 0)
end

response = client.chat(
    parameters: {
        model: "gpt-3.5-turbo", # Required.
        messages: [{ role: "user", content: "Extract Jason is 25 years old"}], # Required.
        temperature: 0.7,
    },
    response_model: UserSchema
)
    
 if response.success?
  # The response is valid, assert each attribute
  user = validation_result.output
  raise 'Name is incorrect' unless user[:name] == "Jason"
  raise 'Age is incorrect'  unless user[:age] == 25

Instructor is mostly a bunch of docs on how to 'think' about the idea. like: https://jxnl.github.io/instructor/concepts/prompting/

[kaiwren]

Somewhat tangentially - I personally tend to favour encapsulating the API call inside a factory method on a PORO domain class that then builds an instance from the response. I then use ActiveModel::Validations for validations on that instance. Something like:

user = User.extract_from('Jason is 25 years old')
if user.valid?
  puts user.name, user.age
else
  puts user.errors.full_messages.join("\n") user.valid?
end

[jxnl]

Somewhat tangentially - I personally tend to favour encapsulating the API call inside a factory method on a PORO domain class that then builds an instance from the response. I then use ActiveModel::Validations for validations on that instance. Something like:

user = User.extract_from('Jason is 25 years old')
if user.valid?
  puts user.name, user.age
else
  puts user.errors.full_messages.join("\n") user.valid?
end

ok, you should check out marvin i think they have the cleanest version of that.

I think its importnat to give the users constrol of the whole messages array.

[sergiobayona]

@jxnl seems like ruby-openai would be able to support it with some patching. Although I wouldn't use dry-validation. I would use ActiveModel.

Something else to consider is that these libs don't output json schema so that'd need to be added... I think.

btw fantastic work you're doing with Instructor. I'm following closely. Here's how it might look like in Ruby:

class UserDetail
  include ActiveModel::Attributes

  attribute :name, :string
  attribute :age, :integer
end

client = OpenAI::Client.new

user = client.chat(
  parameters: {
    model: "gpt-3.5-turbo",
    response_model: UserDetail,
    messages: [{ "role": "user", "content": "Extract Jason is 25 years old" }]
  }
)

RSpec.describe "Attribute Assignment" do
  it "assigns a value to an attribute" do
    expect(user).to be_instance_of(UserDetail)
    expect(user.name).to eq("Jason")
    expect(user.age).to eq(25)
  end
end

[jxnl]

I think that would be awesome. good go everyone.

@jxnl seems like ruby-openai would be able to support it with some patching. Although I wouldn't use dry-validation. I would use ActiveModel.

Something else to consider is that these libs don't output json schema so that'd need to be added... I think.

btw fantastic work you're doing with Instructor. I'm following closely. Here's how it might look like in Ruby:

class UserDetail
  include ActiveModel::Attributes

  attribute :name, :string
  attribute :age, :integer
end

client = OpenAI::Client.new

user = client.chat(
  parameters: {
    model: "gpt-3.5-turbo",
    response_model: UserDetail,
    messages: [{ "role": "user", "content": "Extract Jason is 25 years old" }]
  }
)

RSpec.describe "Attribute Assignment" do
  it "assigns a value to an attribute" do
    expect(user).to be_instance_of(UserDetail)
    expect(user.name).to eq("Jason")
    expect(user.age).to eq(25)
  end
end

That would be awesome. I'm not much of a Ruby developer, but I think it could benefit a lot of folks. We've had a lot of progress on the JavaScript side.

[sergiobayona]

@jxnl I'd like to take a stab at a solution for Ruby for the problem you are solving with Instructor. Would you be interested in collaborating?

[jxnl]

For sure! Are you on Twitter dm me @jxnlco

[sergiobayona]

I think this issue can be closed since instructor-rb solves it. cc @alexrudall