fix(pypi): choose the stock python version before the versioned one (#1637)

[dgdsingen]

for #1637

[jswalton]

👍 for this MR. This is something I really need. Thanks for the work.

[miversen33]

@williamboman is there something preventing this from getting merged?

[williamboman]

Thanks for the PR! This will effectively create the inverse problem where the "stock target" version is too old. The current implementation tries to find a versioned executable that is greater than the stock target.

I believe one thing that could be done to alleviate the issue is to remove python3.12 as a candidate as it seems like it has pretty poor support among pypi packages. It would be optimal if we'd be able to gather information about the supported range of python versions, but there are no convenient out-of-the-box pypi APIs for that afaik.

[dgdsingen]

@williamboman
Thanks for your explanation.
I think what you commented is a very possible scenario.
There is another way to select the appropriate python version by checking the "Requires-Python" metadata value for each pypi package.

• There is the package's "Requires-Python" metadata on here.

• Also can get the package's "Requires-Python" metadata as json.

> ~ curl https://pypi.org/pypi/nginx-language-server/0.8.0/json -sw '' | jq '.info.requires_python'
">=3.8,<3.12"

I think it is possible to match the Python version candidates with "Requires-Python" metadata and select the most recent & appropriate version for the package. What do you think?

About "Requires-Python":
https://packaging.python.org/en/latest/specifications/core-metadata/#requires-python

[williamboman]

Yeah I've seen that field in the API earlier. In order to parse that field properly there'd need to be a PEP440-compatible implementation somewhere. There's a https://docs.rs/pep440_rs/latest/pep440_rs/ crate which could potentially be leveraged in the registry API service, however that API service is opt-outable so a Lua implementation would be preferable. That's pretty much as far I've gotten so far in terms of finding a better solution

[williamboman]

I think a simpler implementation would suffice, I'd wager most version specifiers in that particular field are simple in nature so we don't need to support the full version specifier specification. I'll look into it right now

[williamboman]

@dgdsingen Would you mind trying #1725 (branch feat/pypi-check-supported-python-versions) and see if it behaves in a manner you'd expect it to?

[dgdsingen]

@williamboman
I tested #1725 (branch feat/pypi-check-supported-python-versions) and it works great!

"nginx-language-server v0.8.0" is now installed successfully with python v3.11 (>=3.8,<3.12) on this new branch.

[INFO  토  6/ 1 10:13:45 2024] ...e/nvim/lazy/mason.nvim/lua/mason-core/installer/init.lua:184: Executing installer for Package(name=nginx-language-server) {}
...
[DEBUG 토  6/ 1 10:13:46 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:88: Finding versioned candidates for ">=3.8,<3.12"
...
[DEBUG 토  6/ 1 10:13:46 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:117: Found python3 installation version=3.11.8, executable="python3.11"
...
[DEBUG 토  6/ 1 10:13:50 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:189: pypi: install "nginx-language-server" "0.8.0"
...
[INFO  토  6/ 1 10:13:51 2024] ...e/nvim/lazy/mason.nvim/lua/mason-core/installer/init.lua:245: Installation succeeded for Package(name=nginx-language-server)

And I also tested several other packages I am using (isort, black, ruff-lsp) and it works perfectly 👍👍

[INFO  토  6/ 1 10:16:28 2024] ...e/nvim/lazy/mason.nvim/lua/mason-core/installer/init.lua:184: Executing installer for Package(name=isort) {}
...
[DEBUG 토  6/ 1 10:16:29 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:88: Finding versioned candidates for ">=3.8.0"
...
[DEBUG 토  6/ 1 10:16:29 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:117: Found python3 installation version=3.12.3, executable="python3.12"
...
[DEBUG 토  6/ 1 10:16:31 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:189: pypi: install "isort" "5.13.2"
...
[INFO  토  6/ 1 10:16:32 2024] ...e/nvim/lazy/mason.nvim/lua/mason-core/installer/init.lua:245: Installation succeeded for Package(name=isort)

[INFO  토  6/ 1 10:45:33 2024] ...e/nvim/lazy/mason.nvim/lua/mason-core/installer/init.lua:184: Executing installer for Package(name=black) {}
...
[DEBUG 토  6/ 1 10:45:33 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:88: Finding versioned candidates for ">=3.8"
...
[DEBUG 토  6/ 1 10:45:33 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:117: Found python3 installation version=3.12.3, executable="python3.12"
...
[DEBUG 토  6/ 1 10:45:35 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:189: pypi: install "black" "24.4.2"
...
[INFO  토  6/ 1 10:45:36 2024] ...e/nvim/lazy/mason.nvim/lua/mason-core/installer/init.lua:245: Installation succeeded for Package(name=black)

[INFO  토  6/ 1 10:42:52 2024] ...e/nvim/lazy/mason.nvim/lua/mason-core/installer/init.lua:184: Executing installer for Package(name=ruff-lsp) {}
...
[DEBUG 토  6/ 1 10:42:53 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:88: Finding versioned candidates for ">=3.7"
...
[DEBUG 토  6/ 1 10:42:53 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:117: Found python3 installation version=3.12.3, executable="python3.12"
...
[DEBUG 토  6/ 1 10:42:55 2024] ...zy/mason.nvim/lua/mason-core/installer/managers/pypi.lua:189: pypi: install "ruff-lsp" "0.0.53"
...
[INFO  토  6/ 1 10:42:57 2024] ...e/nvim/lazy/mason.nvim/lua/mason-core/installer/init.lua:245: Installation succeeded for Package(name=ruff-lsp)

Thank you very much for your work!

[williamboman]

Perfect, thanks! I'll close this PR in favour of that one.