Bootstrappable build with modern Gradle

[msgilligan]

We are currently using Gradle 4.4.x on Debian Bookworm for our official builds. Supporting this old version of Gradle while simultaneously supporting building with the latest JDK (currently 21), latest Protobuf plugin, latest JavaFX, etc. is very difficult to maintain. (We are using the Gradle buildscript equivalent of browser feature-testing in JavaScript: lots of Groovy conditionals.)

The reason (I don't links handy, but we can add them to this issue later) that Bookworm (and currently Trixie and Sid) are still on Gradle 4.4.x is the Debian maintainers cannot get a from-source build that is compliant with Debian Free Software Guidelines working. If I recall correctly, part of this issue is with Kotlin which is required to build newer Gradle. (Kotlin on Trixie/Sid is still at 1.3.x as I write this.) I believe there is a now a bootstrappable/reproducible build of Kotlin available but it looks like it has not made it to Debian, yet.

I'm opening this as a tracking issue, so we can keep tabs on developments in the Gradle, Kotlin, and Debian projects. We can also possibly discuss alternatives like Guix or Nix but last I checked they didn't have recent Gradle support, either. Another option might be to switch to Maven. There are more recent builds of Maven on Debian, last I checked.

For a 4-year-old discussion of the problem see the "Java" section of Reproducible Builds Summit, 5th edition Which begins:

Java is a notoriously difficult topic, as far as bootstrapping and reproducibility go.

Sadly, I don't think much has changed in 4 years.

I couldn't find an issue specifically about this problem on the Gradle GitHub so I created gradle/gradle#26516.

[msgilligan]

See PR #3264 and PR #3283 for some of our latest efforts in this saga.

To share a little bit of good news along with these frustrations: we now have a documented reproducible build process using Debian Bookworm in a Docker/Podman container:

• build.Containerfile: reproducible reference build via buildah/podman/docker #3270 (Containerfile for reproducible build)
• build.Containerfile: update reference build to use JDK 17 #3279 (Containerfile upgrade to Bookworm/JDK 17)

[msgilligan]

This looks like the correct link for tracking commits to Debian Gradle: https://salsa.debian.org/java-team/gradle/-/commits/master?ref_type=heads

[msgilligan]

Here's a collection of related links:

Debian

1. https://salsa.debian.org/freexian-team/project-funding/-/blob/master/accepted/2021-08-gradle-enterprise.md
2. https://salsa.debian.org/freexian-team/project-funding/-/issues/19#note_314274
3. https://salsa.debian.org/java-team/kotlin/-/issues/15
4. Debian Bug 926714 - gradle: New upstream version is available - message 66

Kotlin

1. https://blog.jetbrains.com/kotlin/2023/02/reproducible-kotlin-compiler-artifacts/

Gradle

1. Building Gradle without enterprise plugin. gradle/gradle#16439 (comment)
2. Packaging Gradle for Debian gradle/gradle#16512

[msgilligan]

@emorrp1 on the #debian-java IRC/Matrix says:

for bitcoinj, an application which merely uses gradle, the recommendation for Debian packaging at the moment is to completely reimplement the build process in plain maven rather than wait for gradle to work https://wiki.debian.org/Java/Packaging/Maven

@schildbach I have been hinting that we might want to consider, at least for bitcoinj-core.

Here is the mailing list message with the recommendation:
https://lists.debian.org/debian-java/2022/08/msg00010.html

My conclusion so far is that Debian would be better off if we started to use
different build systems whenever possible. Most of the time it would be a
realistic task because many projects upload their artifacts to maven.org and we
could port those Gradle projects to Maven based projects.