Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A few Python fixes #1471

Merged
merged 3 commits into from
May 17, 2024
Merged

A few Python fixes #1471

merged 3 commits into from
May 17, 2024

Conversation

levkk
Copy link
Contributor

@levkk levkk commented May 17, 2024

  1. Show the Python executable when creating the extension (validating Python dependencies).
  2. Print virtualenv error to stderr instead of stdout; this way, it will be captured in Postgres logs.
  3. Add a working Python 3.12 environment, with support for pgml.train(), pgml.embed(), pgml.transform(), pgml.chunk().

@levkk levkk changed the title A few fixes A few Python fixes May 17, 2024
@levkk levkk requested a review from kczimm May 17, 2024 08:14
Copy link
Contributor

@kczimm kczimm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -0,0 +1,97 @@
accelerate==0.30.1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing these are linux python 3.12 requirements. Why not replace requirements.linux.txt with this, and document 3.12 as the Python version required, to avoid sprawling too much. These frozen requirements are for deployments/dockerfiles more than development.

Copy link
Contributor Author

@levkk levkk May 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing these are linux python 3.12 requirements.

Don't have to guess, it's in the PR description.

Why not replace requirements.linux.txt with this, and document 3.12 as the Python version required, to avoid sprawling too much.

ray doesn't run on Python 3.12 and above, which is required by vllm and pgml.transform_stream(). So we actually don't have full support for our extension in Python 3.12.

These frozen requirements are for deployments/dockerfiles more than development.

Well, we have to start somewhere. I need this to run on Python 3.12 locally since I've seen the light and now am using Arch with latest available Python version. Arch users now officially outnumber Ubuntu users, so ...there you go.

@levkk levkk merged commit c321a4c into master May 17, 2024
1 check failed
@levkk levkk deleted the levkk-py312 branch May 17, 2024 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants