Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update openai batch test and workflow #2440

Merged
merged 5 commits into from
May 28, 2024
Merged

update openai batch test and workflow #2440

merged 5 commits into from
May 28, 2024

Conversation

samhita-alla
Copy link
Contributor

@samhita-alla samhita-alla commented May 23, 2024

Tracking issue

flyteorg/flyte#5274

Why are the changes needed?

  • Adds two parameters to the OpenAI workflow, enabling users to specify requests for the file upload and download tasks.
  • Adds metadata to the literal type in the JSON iterator to differentiate it from the FlyteFile transformer, as both return the same literal.
  • Adds try catch to bypass pickling errors https://github.com/flyteorg/flytekit/pull/2356/files

What changes were proposed in this pull request?

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

@samhita-alla samhita-alla marked this pull request as ready for review May 23, 2024 10:13
@samhita-alla samhita-alla changed the title fix openai test & update openai workflow update openai batch test and workflow May 23, 2024
pingsutw
pingsutw previously approved these changes May 24, 2024
Comment on lines 953 to 954
except TypeError: # cannot pickle errors
pass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, when will it happen?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when the input is a generator, i'm seeing this error: Failed with Unknown Exception <class 'TypeError'> Reason: cannot pickle 'generator' object

def jsons():
    for x in [
        {
            "custom_id": "request-1",
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": {
                "model": "gpt-3.5-turbo",
                "messages": [
                    {"role": "system", "content": "You are a helpful assistant."},
                    {"role": "user", "content": "What is 2+2?"},
                ],
            },
        }
    ]:
        yield x

@workflow
def json_iterator_wf(json_val: Iterator[JSON] = jsons()) -> BatchResult:
    return batch(jsonl_in=json_val)

try:
h.update(cloudpickle.dumps(default_inputs))
except TypeError: # cannot pickle errors
pass
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should at the very least log this case, right?

Signed-off-by: Samhita Alla <[email protected]>
@pingsutw pingsutw merged commit 1713765 into master May 28, 2024
44 of 46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants