Binary .local/bin/gitlint missing after update from 0.16.0 to 0.17.0 #250

omarkohl · 2021-11-29T13:37:31Z

After upgrading to the latest release 0.17.0 the executable is gone. Uninstalling and installing again does not help. When installing 0.16.0 all is fine.

After installing 0.17.0 ...

$ pip3 install --user -U gitlint
Collecting gitlint
  Using cached gitlint-0.17.0-py2.py3-none-any.whl (2.7 kB)
Requirement already satisfied: gitlint-core[trusted-deps]==0.17.0 in ./.local/lib/python3.6/site-packages (from gitlint) (0.17.0)
Requirement already satisfied: arrow>=1 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint) (1.2.1)
Requirement already satisfied: Click>=8 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint) (8.0.3)
Requirement already satisfied: sh>=1.13.0 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint) (1.14.2)
Requirement already satisfied: python-dateutil>=2.7.0 in ./.local/lib/python3.6/site-packages (from arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint) (2.8.0)
Requirement already satisfied: typing-extensions in ./.local/lib/python3.6/site-packages (from arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint) (3.7.4.3)
Requirement already satisfied: importlib-metadata in ./.local/lib/python3.6/site-packages (from Click>=8->gitlint-core[trusted-deps]==0.17.0->gitlint) (4.8.1)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.6/site-packages (from python-dateutil>=2.7.0->arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint) (1.11.0)
Requirement already satisfied: zipp>=0.5 in ./.local/lib/python3.6/site-packages (from importlib-metadata->Click>=8->gitlint-core[trusted-deps]==0.17.0->gitlint) (3.6.0)
Installing collected packages: gitlint
Successfully installed gitlint-0.17.0
WARNING: You are using pip version 21.0.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.

... no executable to be found

$ find .local -name *gitlint*
.local/lib/python3.6/site-packages/gitlint_core-0.17.0.dist-info
.local/lib/python3.6/site-packages/gitlint-0.17.0.dist-info

After installing 0.16.0 ...

$ pip3 install --user -U gitlint==0.16.0
...

... everything is fine

$ find .local -name *gitlint*
.local/lib/python3.6/site-packages/gitlint_core-0.17.0.dist-info
.local/lib/python3.6/site-packages/gitlint
.local/lib/python3.6/site-packages/gitlint/files/gitlint
.local/lib/python3.6/site-packages/qa/test_gitlint.py
.local/lib/python3.6/site-packages/qa/__pycache__/test_gitlint.cpython-36.pyc
.local/lib/python3.6/site-packages/gitlint-0.16.0.dist-info
.local/bin/gitlint

$ pip3 --version
pip 21.0.1 from /usr/lib/python3.6/site-packages/pip (python 3.6)

The text was updated successfully, but these errors were encountered:

sigmavirus24 · 2021-11-29T15:56:59Z

I can not reproduce this:

⬢[sigmavirus24@toolbox ~]$ python -m pip install 'gitlint>=0.17'
Defaulting to user installation because normal site-packages is not writeable
Collecting gitlint>=0.17
  Downloading gitlint-0.17.0-py2.py3-none-any.whl (2.7 kB)
Collecting gitlint-core[trusted-deps]==0.17.0
  Downloading gitlint_core-0.17.0-py2.py3-none-any.whl (64 kB)
     |████████████████████████████████| 64 kB 1.7 MB/s
Collecting arrow>=1
  Using cached arrow-1.2.1-py3-none-any.whl (63 kB)
Collecting sh>=1.13.0
  Using cached sh-1.14.2-py2.py3-none-any.whl (40 kB)
Collecting Click>=8
  Using cached click-8.0.3-py3-none-any.whl (97 kB)
Requirement already satisfied: python-dateutil>=2.7.0 in /usr/lib/python3.10/site-packages (from arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint>=0.17) (2.8.1)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.10/site-packages (from python-dateutil>=2.7.0->arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint>=0.17) (1.16.0)
Installing collected packages: sh, Click, arrow, gitlint-core, gitlint
Successfully installed Click-8.0.3 arrow-1.2.1 gitlint-0.17.0 gitlint-core-0.17.0 sh-1.14.2
WARNING: You are using pip version 21.2.3; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python -m pip install --upgrade pip' command.
⬢[sigmavirus24@toolbox ~]$ which gitlint
~/.local/bin/gitlint
⬢[sigmavirus24@toolbox ~]$ gitlint <<EOF
> Ensure this works as intended
>
> Here's a body
> EOF
3: B5 Body message is too short (13<20): "Here's a body"
⬢[sigmavirus24@toolbox ~]$

sigmavirus24 · 2021-11-29T16:00:59Z

Also you seem to have it in your ~/.local/bin as expected too, quoting your initial issue:

$ find .local -name *gitlint*
.local/lib/python3.6/site-packages/gitlint_core-0.17.0.dist-info
.local/lib/python3.6/site-packages/gitlint
.local/lib/python3.6/site-packages/gitlint/files/gitlint
.local/lib/python3.6/site-packages/qa/test_gitlint.py
.local/lib/python3.6/site-packages/qa/__pycache__/test_gitlint.cpython-36.pyc
.local/lib/python3.6/site-packages/gitlint-0.16.0.dist-info
.local/bin/gitlint

It's the last item there. Can you do command -v gitlint? Is your PATH correct?

omarkohl · 2021-11-29T16:49:38Z

I'ts only there after installing version 0.16.0, after installing 0.17.0 it's not there.

pip3 install --user -U gitlint==0.16.0

is the command before the find you quote.

I'm not relying on the PATH and instead using the full .local/bin/gitlint path. As you can see after installing 0.17.0 it's not there:

$ find .local -name *gitlint*
.local/lib/python3.6/site-packages/gitlint_core-0.17.0.dist-info
.local/lib/python3.6/site-packages/gitlint-0.17.0.dist-info

jorisroovers · 2021-11-30T09:46:16Z

So I also tried to reproduce this but couldn't, see shell output below.

To @sigmavirus24's point, I don't think it's possible for shell to find any program/command in ~/.local/bin without it being on your $PATH.

# Simple Ubuntu 20.04 Box
$> cat -p Vagrantfile
Vagrant.configure(2) do |config|
    config.vm.box = "ubuntu/focal64"
end

# Bring up VM, ssh into it
$> vagrant up
$> vagrant ssh

# Install and upgrade pip
$> sudo apt-get update && sudo apt-get install -y python3-pip
$> pip install --user -U pip

# Look at ~/.profile, it adds ~/.local/bin to $PATH, ONLY if that directory exists
$> cat ~/.profile
# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi

# When first logging in, ~/.local/bin didn't exist yet since we hadn't installed anything.
# We have now installed pip, to use it we need to exit+enter the shell or re-source the profile
# to add ~/.local/bin to $PATH
$> source ~/.profile

# Install and run gitlint
$> pip install --user gitlint
$> gitlint --version
gitlint, version 0.17.0

# Note that in order to completely remove the `gitlint` command, you need to uninstall both
# gitlint and gitlint-core
$> pip uninstall gitlint gitlint-core

omarkohl · 2021-11-30T10:12:11Z

It's unfortunate that you can't reproduce it. It's happening on several "SUSE Linux Enterprise Server 15" (aka SLES15), which is probably not your usual test environment.

To @sigmavirus24's point, I don't think it's possible for shell to find any program/command in ~/.local/bin without it being on your $PATH.

I can absolutely guarantee that this is not the issue. I am aware that to execute "gitlint" without a full path ~/.local/bin would have to be on the PATH. But a) I'm only executing "gitlint" WITH the full path and b) ~/.local/bin/gitlint is simply not there, please check the output of my first "find" command (as opposed to the second "find" after installing 0.16.0 where it is there).

jorisroovers · 2021-11-30T10:32:25Z

Hmm, ok. I don't have more time right now though, appreciate if you can help getting to a reproducible scenario, maybe on OpenSUSE (preferably using Vagrant).

I'm intrigued because we did change how we do packaging in 0.17.0, so it's possible that affects installation for some users.

jorisroovers · 2021-11-30T10:59:28Z

Ok, I couldn't resist. I booted an SUSE Linux Enterprise Server 15 SP2 instance on AWS, and still can't reproduce this, see below.

ec2-user@ip-172-31-37-40:~> cat /etc/os-release
NAME="SLES"
VERSION="15-SP2"
VERSION_ID="15.2"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP2"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp2"

ec2-user@ip-172-31-37-40:~> sudo zypper install -y python3-pip git
# removed output
ec2-user@ip-172-31-37-40:~> sudo pip3 install -U pip==21.0.1
# removed output
ec2-user@ip-172-31-37-40:~> python3 --version
Python 3.6.12
ec2-user@ip-172-31-37-40:~> pip3 --version
pip 21.0.1 from /home/ec2-user/.local/lib/python3.6/site-packages/pip (python 3.6)

ec2-user@ip-172-31-37-40:~> pip3 install --user gitlint
Collecting gitlint
  Downloading gitlint-0.17.0-py2.py3-none-any.whl (2.7 kB)
Collecting gitlint-core[trusted-deps]==0.17.0
  Downloading gitlint_core-0.17.0-py2.py3-none-any.whl (64 kB)
     |████████████████████████████████| 64 kB 6.1 MB/s
Collecting sh>=1.13.0
  Downloading sh-1.14.2-py2.py3-none-any.whl (40 kB)
     |████████████████████████████████| 40 kB 11.1 MB/s
Collecting Click>=8
  Downloading click-8.0.3-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 13.0 MB/s
Collecting arrow>=1
  Downloading arrow-1.2.1-py3-none-any.whl (63 kB)
     |████████████████████████████████| 63 kB 5.2 MB/s
Collecting typing-extensions
  Downloading typing_extensions-4.0.0-py3-none-any.whl (22 kB)
Requirement already satisfied: python-dateutil>=2.7.0 in /usr/lib/python3.6/site-packages (from arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint) (2.7.3)
Collecting importlib-metadata
  Downloading importlib_metadata-4.8.2-py3-none-any.whl (17 kB)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.6/site-packages (from python-dateutil>=2.7.0->arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint) (1.14.0)
Collecting zipp>=0.5
  Downloading zipp-3.6.0-py3-none-any.whl (5.3 kB)
Installing collected packages: zipp, typing-extensions, importlib-metadata, sh, Click, arrow, gitlint-core, gitlint
  WARNING: The script gitlint is installed in '/home/ec2-user/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed Click-8.0.3 arrow-1.2.1 gitlint-0.17.0 gitlint-core-0.17.0 importlib-metadata-4.8.2 sh-1.14.2 typing-extensions-4.0.0 zipp-3.6.0
WARNING: You are using pip version 21.0.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.

ec2-user@ip-172-31-37-40:~> find .local -name *gitlint*
.local/lib/python3.6/site-packages/gitlint
.local/lib/python3.6/site-packages/gitlint/files/gitlint
.local/lib/python3.6/site-packages/gitlint_core-0.17.0.dist-info
.local/lib/python3.6/site-packages/gitlint-0.17.0.dist-info
.local/bin/gitlint

ec2-user@ip-172-31-37-40:~> .local/bin/gitlint --version
gitlint, version 0.17.0

# ~/.local/bin/gitlint is NOT in $PATH, but we've already established this is not the issue

Now I'm really out of time for today :-) Appreciate if you can provide the steps to reproduce.

omarkohl · 2021-11-30T11:18:09Z

@jorisroovers thanks for taking the time to look into it! I will try to report more data. (sidenote: I just had a look at your blog, very nice 👍)

omarkohl · 2021-12-01T11:45:53Z

OK, so I got more data 🙂

The problems appears when upgrading from 0.16.0 to 0.17.0 and does NOT appear when installing 0.17.0 on a clean slate. Since 0.17.0 is not cleanly uninstalled, it also appears when re-installing 0.17.0 .

0.16.0 installs/uninstalls cleanly

omark@jenkins-slave-01:~> find .local -name *gitlint*
omark@jenkins-slave-01:~> pip3 install --user -U gitlint==0.16.0
Collecting gitlint==0.16.0
  Using cached gitlint-0.16.0-py2.py3-none-any.whl (82 kB)
Requirement already satisfied: arrow==1.2.0 in ./.local/lib/python3.6/site-packages (from gitlint==0.16.0) (1.2.0)
Requirement already satisfied: Click==8.0.1 in ./.local/lib/python3.6/site-packages (from gitlint==0.16.0) (8.0.1)
Requirement already satisfied: sh==1.14.2 in ./.local/lib/python3.6/site-packages (from gitlint==0.16.0) (1.14.2)
Requirement already satisfied: python-dateutil>=2.7.0 in ./.local/lib/python3.6/site-packages (from arrow==1.2.0->gitlint==0.16.0) (2.8.2)
Requirement already satisfied: typing-extensions in ./.local/lib/python3.6/site-packages (from arrow==1.2.0->gitlint==0.16.0) (4.0.1)
Requirement already satisfied: importlib-metadata in ./.local/lib/python3.6/site-packages (from Click==8.0.1->gitlint==0.16.0) (4.8.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.6/site-packages (from python-dateutil>=2.7.0->arrow==1.2.0->gitlint==0.16.0) (1.11.0)
Requirement already satisfied: zipp>=0.5 in ./.local/lib/python3.6/site-packages (from importlib-metadata->Click==8.0.1->gitlint==0.16.0) (3.6.0)
Installing collected packages: gitlint
  WARNING: The script gitlint is installed in '/home/omark/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed gitlint-0.16.0
WARNING: You are using pip version 21.0.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.

omark@jenkins-slave-01:~> .local/bin/gitlint --version
gitlint, version 0.16.0

omark@jenkins-slave-01:~> pip3 uninstall gitlint
Found existing installation: gitlint 0.16.0
Uninstalling gitlint-0.16.0:
  Would remove:
    /home/omark/.local/bin/gitlint
    /home/omark/.local/lib/python3.6/site-packages/gitlint-0.16.0.dist-info/*
    /home/omark/.local/lib/python3.6/site-packages/gitlint/*
    /home/omark/.local/lib/python3.6/site-packages/qa/*
Proceed (y/n)? y
  Successfully uninstalled gitlint-0.16.0

omark@jenkins-slave-01:~> find .local -name *gitlint*
omark@jenkins-slave-01:~>

0.17.0 does not install/uninstall cleanly

omark@jenkins-slave-01:~> find .local -name *gitlint*
omark@jenkins-slave-01:~> pip3 install --user -U gitlint==0.17.0
Collecting gitlint==0.17.0
  Using cached gitlint-0.17.0-py2.py3-none-any.whl (2.7 kB)
Collecting gitlint-core[trusted-deps]==0.17.0
  Using cached gitlint_core-0.17.0-py2.py3-none-any.whl (64 kB)
Requirement already satisfied: arrow>=1 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.2.0)
Requirement already satisfied: sh>=1.13.0 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.14.2)
Requirement already satisfied: Click>=8 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (8.0.1)
Collecting Click==8.0.3
  Using cached click-8.0.3-py3-none-any.whl (97 kB)
Collecting arrow==1.2.1
  Using cached arrow-1.2.1-py3-none-any.whl (63 kB)
Requirement already satisfied: typing-extensions in ./.local/lib/python3.6/site-packages (from arrow==1.2.1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (4.0.1)
Requirement already satisfied: python-dateutil>=2.7.0 in ./.local/lib/python3.6/site-packages (from arrow==1.2.1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (2.8.2)
Requirement already satisfied: importlib-metadata in ./.local/lib/python3.6/site-packages (from Click==8.0.3->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (4.8.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.6/site-packages (from python-dateutil>=2.7.0->arrow==1.2.1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.11.0)
Requirement already satisfied: zipp>=0.5 in ./.local/lib/python3.6/site-packages (from importlib-metadata->Click==8.0.3->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (3.6.0)
Installing collected packages: Click, arrow, gitlint-core, gitlint
  Attempting uninstall: Click
    Found existing installation: click 8.0.1
    Uninstalling click-8.0.1:
      Successfully uninstalled click-8.0.1
  Attempting uninstall: arrow
    Found existing installation: arrow 1.2.0
    Uninstalling arrow-1.2.0:
      Successfully uninstalled arrow-1.2.0
  WARNING: The script gitlint is installed in '/home/omark/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed Click-8.0.3 arrow-1.2.1 gitlint-0.17.0 gitlint-core-0.17.0
WARNING: You are using pip version 21.0.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.

omark@jenkins-slave-01:~> .local/bin/gitlint --version
gitlint, version 0.17.0

omark@jenkins-slave-01:~> pip3 uninstall gitlint
Found existing installation: gitlint 0.17.0
Uninstalling gitlint-0.17.0:
  Would remove:
    /home/omark/.local/lib/python3.6/site-packages/gitlint-0.17.0.dist-info/*
Proceed (y/n)? y
  Successfully uninstalled gitlint-0.17.0

omark@jenkins-slave-01:~> find .local -name *gitlint*
.local/lib/python3.6/site-packages/gitlint
.local/lib/python3.6/site-packages/gitlint/files/gitlint
.local/lib/python3.6/site-packages/gitlint_core-0.17.0.dist-info
.local/bin/gitlint
omark@jenkins-slave-01:~>

Upgrade from 0.16.0 to 0.17.0 leads to problem

omark@jenkins-slave-01:~> find .local -name *gitlint*
omark@jenkins-slave-01:~> pip3 install --user -U gitlint==0.16.0
Collecting gitlint==0.16.0
  Using cached gitlint-0.16.0-py2.py3-none-any.whl (82 kB)
Requirement already satisfied: sh==1.14.2 in ./.local/lib/python3.6/site-packages (from gitlint==0.16.0) (1.14.2)
Collecting arrow==1.2.0
  Using cached arrow-1.2.0-py3-none-any.whl (62 kB)
Collecting Click==8.0.1
  Using cached click-8.0.1-py3-none-any.whl (97 kB)
Requirement already satisfied: typing-extensions in ./.local/lib/python3.6/site-packages (from arrow==1.2.0->gitlint==0.16.0) (4.0.1)
Requirement already satisfied: python-dateutil>=2.7.0 in ./.local/lib/python3.6/site-packages (from arrow==1.2.0->gitlint==0.16.0) (2.8.2)
Requirement already satisfied: importlib-metadata in ./.local/lib/python3.6/site-packages (from Click==8.0.1->gitlint==0.16.0) (4.8.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.6/site-packages (from python-dateutil>=2.7.0->arrow==1.2.0->gitlint==0.16.0) (1.11.0)
Requirement already satisfied: zipp>=0.5 in ./.local/lib/python3.6/site-packages (from importlib-metadata->Click==8.0.1->gitlint==0.16.0) (3.6.0)
Installing collected packages: Click, arrow, gitlint
  Attempting uninstall: Click
    Found existing installation: click 8.0.3
    Uninstalling click-8.0.3:
      Successfully uninstalled click-8.0.3
  Attempting uninstall: arrow
    Found existing installation: arrow 1.2.1
    Uninstalling arrow-1.2.1:
      Successfully uninstalled arrow-1.2.1
  WARNING: The script gitlint is installed in '/home/omark/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed Click-8.0.1 arrow-1.2.0 gitlint-0.16.0
WARNING: You are using pip version 21.0.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.

omark@jenkins-slave-01:~> pip3 install --user -U gitlint==0.17.0
Collecting gitlint==0.17.0
  Using cached gitlint-0.17.0-py2.py3-none-any.whl (2.7 kB)
Collecting gitlint-core[trusted-deps]==0.17.0
  Using cached gitlint_core-0.17.0-py2.py3-none-any.whl (64 kB)
Requirement already satisfied: sh>=1.13.0 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.14.2)
Requirement already satisfied: arrow>=1 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.2.0)
Requirement already satisfied: Click>=8 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (8.0.1)
Collecting arrow==1.2.1
  Using cached arrow-1.2.1-py3-none-any.whl (63 kB)
Collecting Click==8.0.3
  Using cached click-8.0.3-py3-none-any.whl (97 kB)
Requirement already satisfied: typing-extensions in ./.local/lib/python3.6/site-packages (from arrow==1.2.1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (4.0.1)
Requirement already satisfied: python-dateutil>=2.7.0 in ./.local/lib/python3.6/site-packages (from arrow==1.2.1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (2.8.2)
Requirement already satisfied: importlib-metadata in ./.local/lib/python3.6/site-packages (from Click==8.0.3->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (4.8.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.6/site-packages (from python-dateutil>=2.7.0->arrow==1.2.1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.11.0)
Requirement already satisfied: zipp>=0.5 in ./.local/lib/python3.6/site-packages (from importlib-metadata->Click==8.0.3->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (3.6.0)
Installing collected packages: Click, arrow, gitlint-core, gitlint
  Attempting uninstall: Click
    Found existing installation: click 8.0.1
    Uninstalling click-8.0.1:
      Successfully uninstalled click-8.0.1
  Attempting uninstall: arrow
    Found existing installation: arrow 1.2.0
    Uninstalling arrow-1.2.0:
      Successfully uninstalled arrow-1.2.0
  WARNING: The script gitlint is installed in '/home/omark/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  Attempting uninstall: gitlint
    Found existing installation: gitlint 0.16.0
    Uninstalling gitlint-0.16.0:
      Successfully uninstalled gitlint-0.16.0
Successfully installed Click-8.0.3 arrow-1.2.1 gitlint-0.17.0 gitlint-core-0.17.0
WARNING: You are using pip version 21.0.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.

omark@jenkins-slave-01:~> ./local/bin/gitlint --version
-bash: ./local/bin/gitlint: No such file or directory

omark@jenkins-slave-01:~> find .local -name *gitlint*
.local/lib/python3.6/site-packages/gitlint_core-0.17.0.dist-info
.local/lib/python3.6/site-packages/gitlint-0.17.0.dist-info

Clean installation of 0.17.0 works

omark@jenkins-slave-01:~> find .local -name *gitlint*
omark@jenkins-slave-01:~> pip3 install --user -U gitlint==0.17.0
Collecting gitlint==0.17.0
  Using cached gitlint-0.17.0-py2.py3-none-any.whl (2.7 kB)
Collecting gitlint-core[trusted-deps]==0.17.0
  Using cached gitlint_core-0.17.0-py2.py3-none-any.whl (64 kB)
Requirement already satisfied: Click>=8 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (8.0.3)
Requirement already satisfied: sh>=1.13.0 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.14.2)
Requirement already satisfied: arrow>=1 in ./.local/lib/python3.6/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.2.1)
Requirement already satisfied: python-dateutil>=2.7.0 in ./.local/lib/python3.6/site-packages (from arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (2.8.2)
Requirement already satisfied: typing-extensions in ./.local/lib/python3.6/site-packages (from arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (4.0.1)
Requirement already satisfied: importlib-metadata in ./.local/lib/python3.6/site-packages (from Click>=8->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (4.8.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.6/site-packages (from python-dateutil>=2.7.0->arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.11.0)
Requirement already satisfied: zipp>=0.5 in ./.local/lib/python3.6/site-packages (from importlib-metadata->Click>=8->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (3.6.0)
Installing collected packages: gitlint-core, gitlint
  WARNING: The script gitlint is installed in '/home/omark/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed gitlint-0.17.0 gitlint-core-0.17.0
WARNING: You are using pip version 21.0.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.

omark@jenkins-slave-01:~> .local/bin/gitlint --version
gitlint, version 0.17.0

omarkohl · 2021-12-01T11:51:18Z

So to emphasize:

0.17.0 can't be uninstalled as it should
Upgrading from 0.16.0 to 0.17.0 is a problem (presumably also applies to older version)

As a workaround you can uninstall gitlint 0.16.0 and only then install 0.17.0 . If you want to uninstall 0.17.0 you need some manual cleanup:

pip3 uninstall gitlint
find .local -name *gitlint* -exec rm -rf {} \;

sigmavirus24 · 2021-12-02T00:59:51Z

To uninstall 0.17.0 you need to uninstall gitlint-core

sigmavirus24 · 2021-12-02T01:00:41Z

I suspect this is because the entry_points are only defined in gitlint-core and not in gitlint. I was concerned that might cause issues.

jorisroovers · 2021-12-03T10:15:36Z

Thanks for digging into this @omarkohl, this helped. I've been able to reproduce this in a virtualenv on macOS. I am now convinced this is indeed a gitlint installation bug and not specific to any OS or python/pip version.

Using the verbose flag on pip -v helped me to confirm that this is because pip removes bin/gitlint as part of the gitlint 0.16.0 uninstallation. You can easily see in the output (below), this happens after installation of gitlint and gitlint-core.

As @sigmavirus24 suggested, I also think this happens because we moved the gitlint "binary" (really the entry_point script) to gitlint-core as part of 0.17.0. From pip's point of view, there's a conflict there: 2 different packages are trying to own the same file.

I'm not a pip expert, but I'm guessing pip just uninstalls everything it finds in site-packages/gitlint-0.16.0.dist-info/RECORD, unless referenced from newer versions (and same file checksum or something). Since gitlint 0.17.0 installation record is not referencing bin/gitlint (again, it's part of gitlint-core now), pip will just remove it. Maybe this is not exactly how it works, but I think this is conceptually what's going on.

I think if we were to release 0.17.1, this problem would not occur when upgrading from 0.17.0 to 0.17.1 (or beyond). However, this problem will continue to exist for upgrading from 0.16.0 (and before). Maybe there's ways to fix that by adding special installation steps in setup.py, but IMO we can also just add a note to the Changelog and/or pin a GH issue asking folks to do a clean re-install when upgrading from before 0.16.0 and leave it at that.

For the record, I haven't actually tried this. Appreciate any help getting this confirmed (or new insights :-) )

(.venv)  ✘  /tmp  python --version
Python 3.10.0
(.venv)  /tmp  pip --version
pip 21.3.1 from /private/tmp/.venv/lib/python3.10/site-packages/pip (python 3.10)

(.venv)  /tmp  pip -v install gitlint==0.16.0
Using pip 21.3.1 from /private/tmp/.venv/lib/python3.10/site-packages/pip (python 3.10)
Collecting gitlint==0.16.0
  Using cached gitlint-0.16.0-py2.py3-none-any.whl (82 kB)
Collecting Click==8.0.1
  Using cached click-8.0.1-py3-none-any.whl (97 kB)
Collecting arrow==1.2.0
  Using cached arrow-1.2.0-py3-none-any.whl (62 kB)
Collecting sh==1.14.2
  Using cached sh-1.14.2-py2.py3-none-any.whl (40 kB)
Collecting python-dateutil>=2.7.0
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting six>=1.5
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, python-dateutil, sh, Click, arrow, gitlint
  changing mode of /private/tmp/.venv/bin/gitlint to 755
Successfully installed Click-8.0.1 arrow-1.2.0 gitlint-0.16.0 python-dateutil-2.8.2 sh-1.14.2 six-1.16.0
(.venv)  /tmp  gitlint --version
gitlint, version 0.16.0
(.venv)  ✘  /tmp  find .venv -name '*gitlint*'
.venv/bin/gitlint
.venv/lib/python3.10/site-packages/gitlint-0.16.0.dist-info
.venv/lib/python3.10/site-packages/gitlint
.venv/lib/python3.10/site-packages/gitlint/files/gitlint
.venv/lib/python3.10/site-packages/qa/__pycache__/test_gitlint.cpython-310.pyc
.venv/lib/python3.10/site-packages/qa/test_gitlint.py

(.venv)  /tmp  pip -v install gitlint==0.17.0
Using pip 21.3.1 from /private/tmp/.venv/lib/python3.10/site-packages/pip (python 3.10)
Collecting gitlint==0.17.0
  Using cached gitlint-0.17.0-py2.py3-none-any.whl (2.7 kB)
Collecting gitlint-core[trusted-deps]==0.17.0
  Using cached gitlint_core-0.17.0-py2.py3-none-any.whl (64 kB)
Requirement already satisfied: sh>=1.13.0 in ./.venv/lib/python3.10/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.14.2)
Requirement already satisfied: Click>=8 in ./.venv/lib/python3.10/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (8.0.1)
Requirement already satisfied: arrow>=1 in ./.venv/lib/python3.10/site-packages (from gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.2.0)
Collecting Click>=8
  Using cached click-8.0.3-py3-none-any.whl (97 kB)
Collecting arrow>=1
  Using cached arrow-1.2.1-py3-none-any.whl (63 kB)
Requirement already satisfied: python-dateutil>=2.7.0 in ./.venv/lib/python3.10/site-packages (from arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (2.8.2)
Requirement already satisfied: six>=1.5 in ./.venv/lib/python3.10/site-packages (from python-dateutil>=2.7.0->arrow>=1->gitlint-core[trusted-deps]==0.17.0->gitlint==0.17.0) (1.16.0)
Installing collected packages: Click, arrow, gitlint-core, gitlint
  Attempting uninstall: Click
    Found existing installation: click 8.0.1
    Uninstalling click-8.0.1:
      Removing file or directory /private/tmp/.venv/lib/python3.10/site-packages/click-8.0.1.dist-info/
      Removing file or directory /private/tmp/.venv/lib/python3.10/site-packages/click/
      Successfully uninstalled click-8.0.1
  Attempting uninstall: arrow
    Found existing installation: arrow 1.2.0
    Uninstalling arrow-1.2.0:
      Removing file or directory /private/tmp/.venv/lib/python3.10/site-packages/arrow-1.2.0.dist-info/
      Removing file or directory /private/tmp/.venv/lib/python3.10/site-packages/arrow/
      Successfully uninstalled arrow-1.2.0
  changing mode of /private/tmp/.venv/bin/gitlint to 755
  Attempting uninstall: gitlint
    Found existing installation: gitlint 0.16.0
    Uninstalling gitlint-0.16.0:
      Removing file or directory /private/tmp/.venv/bin/gitlint
      Removing file or directory /private/tmp/.venv/lib/python3.10/site-packages/gitlint-0.16.0.dist-info/
      Removing file or directory /private/tmp/.venv/lib/python3.10/site-packages/gitlint/
      Removing file or directory /private/tmp/.venv/lib/python3.10/site-packages/qa/
      Successfully uninstalled gitlint-0.16.0
Successfully installed Click-8.0.3 arrow-1.2.1 gitlint-0.17.0 gitlint-core-0.17.0

(.venv)  /tmp  gitlint --version
pyenv: gitlint: command not found

(.venv)  ✘  /tmp  find .venv -name '*gitlint*'
.venv/lib/python3.10/site-packages/gitlint_core-0.17.0.dist-info
.venv/lib/python3.10/site-packages/gitlint-0.17.0.dist-info

jorisroovers · 2021-12-03T10:19:53Z

On the uninstall: I think we've all mentioned that gitlint-core needs to also be uninstalled for full cleanup. We can update docs to point this out or further investigate if that can be fixed in setup.py. Separate issue IMO.

This may require extra development documentation due to jorisroovers/gitlint#250, which applies for upgrades.

webknjaz · 2022-11-21T16:04:48Z

@jorisroovers I wish I saw this issue earlier. I know exactly what happens. We've seen this when we were splitting the project under ansible/ansible that was previously published as just ansible on PyPI but got renamed to ansible-base, and then ansible-core. The ansible distribution is also published but with different contents.

TL;DR this is a limitation of pip that is described at pypa/pip#8509.

When you rename a distribution package (the name published on PyPI), but keep that importable package (the folder with Python files that's installed under site-packages), there are two different packages that provide the same files (or partially the same).
There used to be a gitlint/ directory installed into site-packages/ in gitlint==0.16.0. Now, there's none, but a folder with the same name gitlint/ is shipped as a part of gitlint-core which is pulled in by gitlint.
When you do a clean install, pip will resolve the deptree and unpack the dists properly.

When you have gitlint==0.16.0, there's already site-packages/gitlint/ on disk. And pip knows about each file belonging to the said package. During the dependency resolution, it knows that it needs to install gitlint==0.17.0 and gitlint-core==0.17.0 but because of one being a dependency of the other, it knows that gitlint-core should go in first.
So pip goes ahead and unpacks the contents of gitlint-core into site-packages/gitlint/, overwriting the existing files. At this point, pip knows that the files belong to gitlint==0.16.0 and it needs to uninstall it before writing the newer version to disk, it removes these files which got installed from gitlint-core because the gitlint==0.16.0's metadata still says that they belong to this package. And then, it goes ahead and installs gitlint==0.17.0.
This is how you end up with a broken installation.
It is specific to pip upgrades, but not fresh installs. The issue I linked above has a number of suggestions that pip could implement to address this bug (including transactions, similar to other packaging ecosystems), but so far, nobody's working on that.

jorisroovers · 2022-11-22T10:09:14Z

@webknjaz thanks for providing that extra context here, that's insightful!

At this point, I think we should close this particular issue as the root cause and workaround is known, this probably only affects a (very) small fraction of our users and we can't really solve this on our end.

To summarize:

This will only occur when upgrading from gitlint <= 0.16
The solution is to do a clean gitlint install rather than an in-place upgrade: pip uninstall gitlint; pip install gitlint

webknjaz · 2022-11-22T15:35:46Z

The solution is to do a clean gitlint install rather than an in-place upgrade: pip uninstall gitlint; pip install gitlint

More precisely, it should be pip uninstall gitlint gitlint-core; pip install gitlint which would also take care of a semi-broken install.

At this point, I think we should close this particular issue

At Ansible, we've opted for documenting this in the FAQ: https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_4.html#other / https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.10.html#known-issues.
Maybe, gitlint could do the same? Probably, we could change the issue label to docs at this point.

jorisroovers · 2022-11-23T09:47:43Z

Sounds good - I'll keep this issue open then as a reminder to document this.

I think we probably want a net new page for this in the docs (FAQ perhaps). I was looking for a similar spot to put the deprecation docs related to #254, and then ended up inlining them in the configuration docs instead. Perhaps we can also make a reference to those docs in that new page.

Happy to take a PR :)

jorisroovers added the bug User-facing bugs label Dec 3, 2021

omarkohl changed the title ~~Binary .local/bin/gitlint missing after installing 0.17.0~~ Binary .local/bin/gitlint missing after update from 0.16.0 to 0.17.0 Dec 6, 2021

neiljp added a commit to neiljp/zulip-terminal that referenced this issue Mar 25, 2022

requirements[dev_helper]: Upgrade gitlint from >=0.10 to >=0.17.

09db5dc

This may require extra development documentation due to jorisroovers/gitlint#250, which applies for upgrades.

neiljp added a commit to zulip/zulip-terminal that referenced this issue Mar 28, 2022

requirements[dev_helper]: Upgrade gitlint from >=0.10 to >=0.17.

7a899ad

This may require extra development documentation due to jorisroovers/gitlint#250, which applies for upgrades.

jorisroovers added docs Documentation related items and removed bug User-facing bugs labels Nov 23, 2022

jorisroovers added this to the 0.20.0 milestone Mar 9, 2023

jorisroovers self-assigned this Mar 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Binary .local/bin/gitlint missing after update from 0.16.0 to 0.17.0 #250

Binary .local/bin/gitlint missing after update from 0.16.0 to 0.17.0 #250

omarkohl commented Nov 29, 2021 •

edited

sigmavirus24 commented Nov 29, 2021

sigmavirus24 commented Nov 29, 2021

omarkohl commented Nov 29, 2021 •

edited

jorisroovers commented Nov 30, 2021 •

edited

omarkohl commented Nov 30, 2021 •

edited

jorisroovers commented Nov 30, 2021

jorisroovers commented Nov 30, 2021 •

edited

omarkohl commented Nov 30, 2021

omarkohl commented Dec 1, 2021

omarkohl commented Dec 1, 2021

sigmavirus24 commented Dec 2, 2021

sigmavirus24 commented Dec 2, 2021

jorisroovers commented Dec 3, 2021

jorisroovers commented Dec 3, 2021 •

edited

webknjaz commented Nov 21, 2022

jorisroovers commented Nov 22, 2022 •

edited

webknjaz commented Nov 22, 2022

jorisroovers commented Nov 23, 2022

Binary .local/bin/gitlint missing after update from 0.16.0 to 0.17.0 #250

Binary .local/bin/gitlint missing after update from 0.16.0 to 0.17.0 #250

Comments

omarkohl commented Nov 29, 2021 • edited

After installing 0.17.0 ...

... no executable to be found

After installing 0.16.0 ...

... everything is fine

sigmavirus24 commented Nov 29, 2021

sigmavirus24 commented Nov 29, 2021

omarkohl commented Nov 29, 2021 • edited

jorisroovers commented Nov 30, 2021 • edited

omarkohl commented Nov 30, 2021 • edited

jorisroovers commented Nov 30, 2021

jorisroovers commented Nov 30, 2021 • edited

omarkohl commented Nov 30, 2021

omarkohl commented Dec 1, 2021

0.16.0 installs/uninstalls cleanly

0.17.0 does not install/uninstall cleanly

Upgrade from 0.16.0 to 0.17.0 leads to problem

Clean installation of 0.17.0 works

omarkohl commented Dec 1, 2021

sigmavirus24 commented Dec 2, 2021

sigmavirus24 commented Dec 2, 2021

jorisroovers commented Dec 3, 2021

jorisroovers commented Dec 3, 2021 • edited

webknjaz commented Nov 21, 2022

jorisroovers commented Nov 22, 2022 • edited

webknjaz commented Nov 22, 2022

jorisroovers commented Nov 23, 2022

omarkohl commented Nov 29, 2021 •

edited

omarkohl commented Nov 29, 2021 •

edited

jorisroovers commented Nov 30, 2021 •

edited

omarkohl commented Nov 30, 2021 •

edited

jorisroovers commented Nov 30, 2021 •

edited

jorisroovers commented Dec 3, 2021 •

edited

jorisroovers commented Nov 22, 2022 •

edited