Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

language/python: order args for virtualenv_install_with_resources #16817

Merged

Conversation

cho-m
Copy link
Member

@cho-m cho-m commented Mar 5, 2024

Add without, start_with, and end_with to allow basic control over the order that resources are installed so that the situations where we have to split up virtualenv_install_with_resources is reduced.

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew style with your changes locally?
  • Have you successfully run brew typecheck with your changes locally?
  • Have you successfully run brew tests with your changes locally?

In homebrew-core, we've often have to running separate venv commands whenever there is an ordering problem or there are some extra resources that we don't want in venv.

In this PR, I am looking into expanding DSL to handle some of these cases.

Examples usage can include:

  • jupyterlab:
    @@ -519,13 +519,11 @@ class Jupyterlab < Formula
       end
    
       def install
    -    venv = virtualenv_create(libexec, python3)
         ENV["JUPYTER_PATH"] = etc/"jupyter"
    
         # install packages into virtualenv and link all jupyter extensions
         skipped = %w[hatch-jupyter-builder hatch-nodejs-version jupyterlab-pygments notebook]
    -    venv.pip_install resources.reject { |r| skipped.include? r.name }
    -    venv.pip_install_and_link buildpath
    +    venv = virtualenv_install_with_resources without: skipped
         bin.install_symlink (libexec/"bin").glob("jupyter*")
  • awscli
    @@ -133,10 +133,7 @@ class Awscli < Formula
         end
    
         # The `awscrt` resource requires `setuptools` & `wheel`, so they must be installed first
    -    venv = virtualenv_create(libexec, "python3.11", system_site_packages: false)
    -    venv.pip_install resources.reject { |r| r.name == "awscrt" }
    -    venv.pip_install resource("awscrt")
    -    venv.pip_install_and_link buildpath
    +    virtualenv_install_with_resources system_site_packages: false, end_with: "awscrt"

I added an exception in case that resource name doesn't exist, but can remove if we want to just ignore these.


EDIT: Probably need to view test diff without whitespace as I ended up indenting existing Language::Python::Virtualenv::Virtualenv (https://github.com/Homebrew/brew/pull/16817/files?diff=unified&w=1)

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea! Some more comments, none of which are blocking.

Library/Homebrew/language/python.rb Outdated Show resolved Hide resolved
Library/Homebrew/test/language/python/virtualenv_spec.rb Outdated Show resolved Hide resolved
Copy link
Contributor

@apainintheneck apainintheneck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I like the the DSL. I left a few nits though.

The error message will be weird if someone decides to add the same name to multiple of these lists but then again, who would do that?

Library/Homebrew/language/python.rb Outdated Show resolved Hide resolved
Library/Homebrew/language/python.rb Outdated Show resolved Hide resolved
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale No recent activity label Mar 28, 2024
@cho-m cho-m removed the stale No recent activity label Mar 28, 2024
@cho-m
Copy link
Member Author

cho-m commented Apr 13, 2024

I'll merge suggestions and then adjust remaining parts after testing out logic.

@cho-m cho-m force-pushed the virtualenv-basic-resource-order-args branch from 54f3367 to e333cfc Compare April 13, 2024 23:48
@cho-m
Copy link
Member Author

cho-m commented Apr 14, 2024

Need another review as I refactored logic into separate method so want to make sure if this is good. I was trying out other loop logic from my review comment, but Sorbet doesn't seem capable of keeping track of types in that situation. I do think current logic is more readable than my other attempts.

Copy link
Contributor

@apainintheneck apainintheneck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this is easier to understand than before. The error message is a bit less specific but it should be clear enough.

Library/Homebrew/language/python.rb Outdated Show resolved Hide resolved
Add `without`, `start_with`, and `end_with` to allow basic control over
the order that resources are installed so that the situations where we
have to split up `virtualenv_install_with_resources` is reduced.

Co-authored-by: Kevin <[email protected]>
Co-authored-by: Mike McQuaid <[email protected]>
Signed-off-by: Michael Cho <[email protected]>
@cho-m cho-m force-pushed the virtualenv-basic-resource-order-args branch from e333cfc to ecb7dab Compare April 15, 2024 02:22
Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me, thanks again @cho-m!

@MikeMcQuaid MikeMcQuaid merged commit c44bf6f into Homebrew:master Apr 15, 2024
25 checks passed
@cho-m cho-m deleted the virtualenv-basic-resource-order-args branch April 15, 2024 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants