Inconsistent PATH set with hatch run

[jlorieau]

I've encountered an unexpected behavior with hatch run (1.10.0) on MacOS X (Sonoma 14.4 on arm).

I've created a docs environment for sphinx, following the hatch configuration from Jupyter notebook. When I enter the shell, the environment script (activate) gets properly sourced and the PATH gets correctly set.

$ hatch shell docs
$ echo $PATH
/Users/jlorieau/Code/python/xamin/xamin/.hatch/docs/bin:...
$ which python
/Users/jlorieau/Code/python/xamin/xamin/.hatch/docs/bin/python3
$ which sphinx-build
/Users/jlorieau/Code/python/xamin/xamin/.hatch/docs/bin/sphinx-build

Likewise, when I exit the environment and use hatch -e docs run, I get the correct python path.

$ hatch -e docs run which python
/Users/jlorieau/Code/python/xamin/xamin/.hatch/docs/bin/python

But other scripts, which are also available on my system, point outside of the virtualenv.

$ hatch -e docs run which sphinx-build
/opt/homebrew/bin/sphinx-build

(I expect this to output /Users/jlorieau/Code/python/xamin/xamin/.hatch/docs/bin/sphinx-build).

It appears that the virtual environment hasn't prepended the virtual environment PATH when using hatch -e docs run echo $PATH.

I haven't read through the hatch source, but I'm guessing that hatch run doesn't source the environment (in my case, source .hatch/docs/bin/activate) to properly prepend the PATH.

Presumably, Jupyter notebook hasn't encountered this problem when running their script command: hatch run docs:build. I believe that I'm missing a configuration setting or something--is that the case?

Nevertheless, this might be a bug or an unsupported feature I could not find in the documentation.

Thank you!

[ofek]

hatch run docs:echo $PATH wouldn't show you the modification because your shell would expand the variable so instead you would have to run something like hatch run docs:sh -c 'echo $PATH'

Can you try that to confirm?

[jlorieau]

Makes sense, thank you.
Running the command within the new shell shows the virtualenv in the PATH, but it's 7th on the list. It looks like a bunch of the paths I prepended from my zshrc were added before it, including ~/.local/bin, /opt/homebrew/bin and so on.

[jlorieau]

Note also that running the following command places the $VIRTUAL_ENV/bin path later (8th place, not 7th) in the list.

$ hatch run docs:python -c "import os;print(os.environ['PATH'])"
/Users/jlorieau/.local/bin:..:/Users/jlorieau/Code/python/xamin/xamin/.hatch/docs/bin:...

[ofek]

I think you have to use the argument that Hatch uses shell=True

[jlorieau]

Thanks, I got mixed up with the tuple/list version. Running as you suggested shows that it should work. In this case, I'm running from the default virtual env named 'xamin', so that one is at the top of the path until I prepend the virtual env docs to the environment:

>>> subprocess.run('echo $PATH', shell=True)
/Users/jlorieau/Code/python/xamin/xamin/.hatch/xamin/bin:...
>>> os.environ["PATH"] = "/Users/jlorieau/Code/python/xamin/xamin/.hatch/docs/bin:" + os.environ["PATH"]
>>> subprocess.run('echo $PATH', shell=True)
/Users/jlorieau/Code/python/xamin/xamin/.hatch/docs/bin:/Users/jlorieau/Code/python/xamin/xamin/.hatch/xamin/bin: ...

(Sorry about my path depths)

I also tried running hatch from a bash shell, in case one of my zsh plugins was being too clever, and I have the same problem:

$ hatch run docs:sh -c 'echo $PATH' 
/Users/jlorieau/.local/bin: ... :/Users/jlorieau/Code/python/xamin/xamin/.hatch/docs/bin:/Users/jlorieau/Code/python/xamin/xamin/.hatch/xamin/bin: ... 

The docs virtualenv is 8th in my list.

[ofek]

Thanks for debugging! I wonder if this could be reproduced in a Dockerfile. Tomorrow I leave for Pittsburgh to present at PyCon on Friday but when I return I can think about this further.

[jlorieau]

Thank you for the note.

I will be traveling over the next 20 or so days too. I'll aim to make a Dockerfile to reproduce the error, and it may include cloning my dotfiles.

In any case, I have a reasonable workaround (see above) so it's not critical for my work.

Have a good conference.