Checksum of GitHub releases' tarballs changing (v6.0.2)?

[Antiz96]

Hi,

I'm maintaining the molecule Arch Linux package and we recently noticed that the checksum of the GitHub release v6.0.2 tarball changed at some point between Sep 03 (the first time we built that release) and Oct 13 (the day we noticed the checksum change).

Here's the rebuild we made with the updated checksum for the v6.0.2 tarball once we noticed the change.

We try our best to have reproducible packages builds and such changes makes it harder to reach.
Do you know what could have triggered this checksum change in the v6.0.2 tarball? If so, is it intentional/expected?

I remain available if you need any additional information! :)

[ssbarnea]

Anyone using github tar.gz checksums should be aware that there is no guarantee regarding them remaining the same. At any point in time, Github can change them.

I know about this as I seen a similar problem with homebrew which can do the same in some cases.

Read https://github.com/orgs/community/discussions/45830#discussioncomment-4823531 for more hints.

So, not on purpose, outside our control. How often to expect it?... Ask Github. My guess is that not very often.

[dvzrv]

Anyone using github tar.gz checksums should be aware that there is no guarantee regarding them remaining the same. At any point in time, Github can change them.

This is not true at least before beginning of next year. And it's questionable whether they can pull it off without breaking literally all downstream distributions.

FTR: Github long reverted their change to their tarball generation functionality from this year, exactly because they were breaking everyone (see https://github.blog/changelog/2023-01-30-git-archive-checksums-may-change/).

As a sidenote: We have moved to using upstream provided tarballs, because PyPI is unreliable and sdist tarballs are poorly defined (https://rfc.archlinux.page/0020-sources-for-python-packaging/).

So, not on purpose, outside our control. How often to expect it?... Ask Github. My guess is that not very often.

As pointed out in ansible/ansible-lint#2781 this is not outside of your control, but completely depending on the configuration of your tooling.
For ansible-lint this commit broke the tarball reproducibility.

FTR: I maintain around 1000 packages for Arch Linux and a very large subset of them use github and in a very large subset of those I use auto-generated source tarballs (because there are no other). This happens without a single problem (unless someone retags) and it is only projects such as molecule and ansible-lint that break (because of their git-archive configuration).

To reiterate: The issue exists, because molecule and ansible-lint add git-archival.txt files (which btw are not needed for setuptools-scm, overrides via environment variables such as SETUPTOOLS_SCM_PRETEND_VERSION exist for this purpose).
Every time the release branch changes after a release, the contents of the git-archival.txt file changes because of that.

diffoscope old/molecule-6.0.2.tar.gz molecule-6.0.2.tar.gz
 |#                                                                                                                              |  N/A%  None  ETA:      N/A --- old/molecule-6.0.2.tar.gz
+++ molecule-6.0.2.tar.gz
├── molecule-6.0.2.tar
│ ├── file list
│ │ @@ -5,15 +5,15 @@
│ │  -rw-rw-r--   0 root         (0) root         (0)     2199 2023-08-30 10:19:47.000000 molecule-6.0.2/.config/molecule.spec
│ │  drwxrwxr-x   0 root         (0) root         (0)        0 2023-08-30 10:19:47.000000 molecule-6.0.2/.config/molecule/
│ │  -rw-rw-r--   0 root         (0) root         (0)       41 2023-08-30 10:19:47.000000 molecule-6.0.2/.config/molecule/config.yml
│ │  -rw-rw-r--   0 root         (0) root         (0)       58 2023-08-30 10:19:47.000000 molecule-6.0.2/.config/requirements-docs.txt
│ │  -rw-rw-r--   0 root         (0) root         (0)      416 2023-08-30 10:19:47.000000 molecule-6.0.2/.config/requirements-test.txt
│ │  -rw-rw-r--   0 root         (0) root         (0)       24 2023-08-30 10:19:47.000000 molecule-6.0.2/.config/requirements-testinfra.txt
│ │  -rw-rw-r--   0 root         (0) root         (0)      226 2023-08-30 10:19:47.000000 molecule-6.0.2/.config/requirements.in
│ │ --rw-rw-r--   0 root         (0) root         (0)       37 2023-08-30 10:19:47.000000 molecule-6.0.2/.git_archival.txt
│ │ +-rw-rw-r--   0 root         (0) root         (0)       23 2023-08-30 10:19:47.000000 molecule-6.0.2/.git_archival.txt
│ │  -rw-rw-r--   0 root         (0) root         (0)     3277 2023-08-30 10:19:47.000000 molecule-6.0.2/.gitattributes
│ │  drwxrwxr-x   0 root         (0) root         (0)        0 2023-08-30 10:19:47.000000 molecule-6.0.2/.github/
│ │  -rw-rw-r--   0 root         (0) root         (0)       64 2023-08-30 10:19:47.000000 molecule-6.0.2/.github/CODEOWNERS
│ │  -rw-rw-r--   0 root         (0) root         (0)      162 2023-08-30 10:19:47.000000 molecule-6.0.2/.github/CODE_OF_CONDUCT.md
│ │  drwxrwxr-x   0 root         (0) root         (0)        0 2023-08-30 10:19:47.000000 molecule-6.0.2/.github/ISSUE_TEMPLATE/
│ │  -rw-rw-r--   0 root         (0) root         (0)     1042 2023-08-30 10:19:47.000000 molecule-6.0.2/.github/ISSUE_TEMPLATE/bug_report.md
│ │  -rw-rw-r--   0 root         (0) root         (0)     2050 2023-08-30 10:19:47.000000 molecule-6.0.2/.github/ISSUE_TEMPLATE/config.yml
│ ├── molecule-6.0.2/.git_archival.txt
│ │ @@ -1 +1 @@
│ │ -ref-names: HEAD -> main, tag: v6.0.2
│ │ +ref-names: tag: v6.0.2

Please realize, that this is maximum painful for redistributors, who work on this in their free time.

From my perspective there the following ways of dealing with this (both can even be applied side-by-side):

• removing git-archival.txt and git-attributes.txt (as noted, setuptools-scm doesn't need them)
• adding source tarballs to releases, so that there are source tarballs, that never change (also if github ever decides to mess up their tarball auto generation again)

[Torxed]

How often to expect it?... Ask Github. My guess is that not very often.

Often enough. Especially if you consider the bigger picture. Cumulative issues across many projects - that is the real issue.. Which is why fixing them is a priority.

We need to free up time to do more meaningful open source work — so my question is what would be the harm in applying the suggested fixes?

[ssbarnea]

If you want immutable archives, use PyPi ones as they are like this by definition. GitHub based ones may change, either due to GitHub changes or because we might do a retagging at some point. We don't plan to do that, but we also offer no guarantees on immutability of the uploaded artifacts, we never did. In fact these artifacts are in general a blend of auto-generated and managed by github and some manually uploaded, and there is no clear way to distinguish between them.

The original post was bit misleading in regards to what setuptool-scm needs, as it does need those files. The reality is that molecule itself had an outdated file, one that will be fixed by #4071 - I hope that this will improve your experience.

We will not get rid of .git_archival.txt as that is mandatory in order to allow installation from github archive of a specific version while still keeping the version information. Use of the environment variable hack is known to us but is not usable in all cases we need to support, thus is why we relied on archive approach.

To summarize: for immutability with checksums, please rely on pypi.