poetry install run under pre-commit hook does not apply changes.

[thisiswhereitype]

Description

pre-commit run poetry-install passed but acted as no-op.

The expected package was not in the venv.
A regular: poetry install noticed the missing package.

Workarounds

Manually poetry install.

Poetry Installation Method

pipx

Operating System

Win 10

Poetry Version

v1.8.2

Poetry Configuration

> poetry config --list
cache-dir = "~\\AppData\\Local\\pypoetry\\Cache"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}\\virtualenvs"  # ~\AppData\Local\pypoetry\Cache\virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true

Python Sysconfig

> python -m sysconfig 
Platform: "win-amd64"
Python version: "3.12"
Current installation scheme: "venv"

Paths:
        data = "<repo>\.venv"
        include = "~\scoop\apps\python\3.12.3\Include"
        platinclude = "~\scoop\apps\python\3.12.3\Include"
        platlib = "<repo>\.venv\Lib\site-packages"
        platstdlib = "<repo>\.venv\Lib"
        purelib = "<repo>\.venv\Lib\site-packages"
        scripts = "<repo>\.venv\Scripts"
        stdlib = "~\scoop\apps\python\3.12.3\Lib"

Variables:
        BINDIR = "<repo>\.venv\Scripts"
        BINLIBDEST = "<repo>\.venv\Lib"
        EXE = ".exe"
        EXT_SUFFIX = ".cp312-win_amd64.pyd"
        INCLUDEPY = "~\scoop\apps\python\3.12.3\Include"
        LIBDEST = "~\scoop\apps\python\3.12.3\Lib"
        TZPATH = ""
        VERSION = "312"
        VPATH = "..\.."
        abiflags = ""
        base = "<repo>\.venv"
        exec_prefix = "<repo>\.venv"
        installed_base = "~\scoop\apps\python\3.12.3"
        installed_platbase = "~\scoop\apps\python\3.12.3"
        platbase = "<repo>\.venv"
        platlibdir = "DLLs"
        prefix = "<repo>\.venv"
        projectbase = "~\scoop\apps\python\3.12.3"
        py_version = "3.12.3"
        py_version_nodot = "312"
        py_version_nodot_plat = "312"
        py_version_short = "3.12"
        srcdir = "~\scoop\apps\python\3.12.3"
        userbase = "%HOMEDRIVE%%HOMEPATH%\.python"

Example pyproject.toml

[virtualenvs]
in-project = true

Precommit:

default_install_hook_types: [pre-commit, post-checkout]
default_stages: [pre-commit]
fail_fast: true
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
  - repo: https://github.com/python-poetry/poetry
    rev: "1.8.0"
    hooks:
      - id: poetry-check
      - id: poetry-lock
      - id: poetry-export
      - id: poetry-install

Repo Transcript

> poetry install --dry-run
Installing dependencies from lock file

Package operations: 1 install, 0 updates, 0 removals, 179 skipped

  - Installing pip (23.3.1): Skipped for the following reason: Already installed
  ...
  - Installing watermark (2.4.3)
  - Installing voluptuous (0.13.1): Skipped for the following reason: Already installed
  - Installing wheel (0.41.3): Skipped for the following reason: Already installed
  - Installing wcwidth (0.2.9): Skipped for the following reason: Already installed
  - Installing yarl (1.9.2): Skipped for the following reason: Already installed
  - Installing widgetsnbextension (4.0.9): Skipped for the following reason: Already installed
  - Installing tzdata (2023.3): Skipped for the following reason: Already installed
  - Installing zipp (3.18.1): Skipped for the following reason: Already installed
  - Installing zc-lockfile (3.0.post1): Skipped for the following reason: Already installed

Installing the current project: <package> (0.10.0)

> git checkout
Your branch is up to date with 'origin/<branch>'.
poetry-install...........................................................Passed
dvc checkout.............................................................Passed
> python -c "import watermark"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'watermark'
> poetry env list
.venv (Activated)

[dimbleby]

I expect you have not read the docs telling you about the extra things you need to do when using this hook https://python-poetry.org/docs/pre-commit-hooks#poetry-install

[thisiswhereitype]

I expect you have not read the docs telling you about the extra things you need to do when using this hook https://python-poetry.org/docs/pre-commit-hooks#poetry-install

Thanks - I've added a transcript of the output and clarified the hooks in the pre-commit file. Am I missing something else? The docs say something about args but that pre-commit file should be applying to the venv?

[dimbleby]

And is the python you are running at python -c "import watermark" the same as the python from the virtual environment that poetry is running? Probably you just need to activate the virtual environment (or poetry run ...)

[thisiswhereitype]

And is the python you are running at python -c "import watermark" the same as the python from the virtual environment that poetry is running? Probably you just need to activate the virtual environment (or poetry run ...)

Yes - they are run back to back in the same terminal, dvc wouldn't be found outside this venv. I added the poetry env list.

[dimbleby]

not a pre-commit user myself, but so far as I know pre-commit likes to run in its own virtual environment.

trying it myself it seems as though it is attempting to install the project packages in the virtual environment containing the pre-commit poetry: which is both doing nothing for the virtual environment that you care about - and likely to fail altogether when the poetry requirements clash with the project requirements.

seems completely broken to me, perhaps we should try asking someone from #8327 what we have misunderstood @alsmnn @radoering

[radoering]

I do not use the poetry pre-commit hooks myself, but trying it I can confirm that this hook seems broken - and I wonder how/if it has ever worked.

[alsmnn]

Ackchyually it installs all your dependencies, but in the virtual environment determined by pre-commit. You can verify that by looking under:
~/.cache/pre-commit/repo{some_hash_value}/py_env-python{your_python_version}/lib/python{your_python_version}/site-packages/.
If you want to run pre-commit against your local poetry, you can use this config:

default_install_hook_types: [pre-commit, post-checkout, post-merge]
repos:
  - repo: local
    hooks:
      - id: poetry-install
        name: poetry-install
        args: ["--sync"]
        language: system
        entry: poetry install
        stages: [post-checkout, post-merge]
        pass_filenames: false
        always_run: true
        verbose: true

The verbose: true statement is helpful to see what poetry is actually doing.
I don't know why we thought it would work correctly in the past. It must have slipped my attention, that it just updates the dependencies in the virtual environment which was provided by pre-commit. I will open a PR in the next few days and see if I can correct that mistake.
Thanks for finding that bug!