Getting the next release ready

[krobelus]

I'm selfishly motivated to speed up the release (and feedback) cycle.
At the same time, there are some changes I'm merging to master only because there is no near release planned. But doing that might delay a release which I don't want.

I'm not sure

1. when the release will happen
2. what needs to be done to make that happen (not including nice-to-have improvements)

Some points raised before:

1. We might have latent crashes (from the Rust port).
   Fair point, although after a few months of soaking we get diminishing returns from beta-testing.
   The best thing to do here is read the relevant source code changes, I'm happy to do that.
2. Some distributions require packaging (instead of delegating to Cargo).
   For example Debian policy says
   "Application crates must have Build-Depends on the Rust library and feature packages needed to build, with appropriate version constraints corresponding to the version predicates of that crate's Cargo dependencies."
   We need to at least tag releases of our unreleased dependencies.
   It is not clear to me me what else we need to do here but perhaps we should
   provide packages in our Debian PPAs (as librust-pcre2-widechar, librust-fast-float, librust-hexponent, librust-printf-compat).
   Is that enough? Do we also want to drive the effort to add them to the official Debian repos?
   I know that crates.io rejects dependencies that are not on crates.io but I don't think this matters to us at this point? Either way we can easily publish those dependencies on crates.io.

In our concrete situtation, I suggest we release master (including any additions) in 1-3 months.
Alternatively we could release all but recent risky changes now (but it's hard to draw the line).
Regardless, we need to be prepared to follow-up with a patch release.

Here's a todo list:

• release our git-dependencies, and perhaps build distro packages
• Make our PPAs work again if/where desired.
  Nightlies are currently broken, see https://build.opensuse.org/project/show/shells:fish:nightly:master
  • On distros that don't package all Rust dependencies, we'll want to use cargo vendor so the OBS builder can build without internet access.
    It looks like we provide packages for OpenSUSE, Fedora, Debian, CentOS.
    Which ones do we want to keep supporting here?
  • CentOS is discontinued, can we drop that?
  • For Debian only versions >=13 has rustc>=1.67 so I'd require Debian 13. (I tried upgrading to that but OBS doesn't give great errors)
  • For Fedora, getting rid of git dependencies is not encouraged but not required so that should be easy to get working.
    • Some Fedora versions don't work (armv7l has a weird error) and powerpc doesn't have rustc>=1.67, can we drop/postpone them?
  • openSUSE_Leap_42.3 doesn't have recent rustc, so drop it?
  • openSUSE 15.4 seems to have a missing dependency on gcc-13, that should be easy to fix.
• Anything else? I'm not aware of other blockers.

I'm happy to make the changelog additions.
Since work on the changelog is shared and we use it to inform decisions about making a release I tend to err on the side of adding too much, should probably stop that.

---

Another comment was that changes (some time ago) were not impactful enough to compensate for the risk.
I think this is an important consideration and I don't necessarily disagree but I'd like to add that impact is inherently subjective.
Going forward -- and I think everyone agrees -- I'd like to make more frequent releases (perhaps biannually), even if there is no flagship feature.
The little usability improvements can have a great impact if someone uses them daily.
Sometimes high-risk changes can be tested on side branches or with feature flags.

[faho]

Another comment was that changes (some time ago) were not impactful enough to compensate for the risk.

I think we've passed that point, it's a sizable release now.

Going forward -- and I think everyone agrees -- I'd like to make more frequent releases (perhaps biannually), even if there is no flagship feature.

Typically yes, but this one is a bit special because of how much was changed internally.

CentOS is discontinued, can we drop that?

Technically CentOS 7 is still supported until 2024-06-30 - CentOS 8 was discontinued early.

The issue here is that that's also RHEL. We might be able to get one of the other RHEL forks?

I know that crates.io rejects dependencies that are not on crates.io but I don't think this matters to us at this point? Either way we can easily publish those dependencies on crates.io.

The ideal solution would be to upstream changes we need. I know we're stalled at least on pcre2, and I'm not sure how well maintained these other ones are.

We can ask the terminfo people to do a release.

Some Fedora versions don't work (armv7l has a weird error) and powerpc doesn't have rustc>=1.67, can we drop/postpone them?

I don't believe these are all that important, and frankly if an architecture lags rust by that much (it's 10 releases behind) I don't think it's very healthy anyway.

---

One big thing is: We do not need to follow distro policy in our PPAs.

We don't have to split out the dependencies for the debian PPA, we don't have to use the distro rustc.

Can we just use rustup on OBS?

[zanchey]

I already have the infrastructure for building and uploading vendor tarballs working, it's just not committed yet until I'm sure it works end-to-end. I can push a temporary version somewhere if you want to see it. I think this is fine for building binary packages, although it would be good to have our patches to dependencies committed upstream to make it easier for native packages to be build (eg Fedora, Debian).

Rustup on OBS is not really an option. The builders don't have network access, and uploading a package containing a bunch of binaries is forbidden by policy (though the OBS Rust packages use the bootstrap binaries which is considered acceptable).

Fedora are pretty aggressive about getting new versions into supported Fedora releases. I leave the builds turned on as I think they're a useful canary for changes but I don't expect them to get much use. We don't link to OBS from the homepage for Fedora for this reason.

Dropping CentOS 7 is fine by me. Backporting CMake is required to get it working, and I'm happy to do that but it requires a Fedora VM which is another yak to shave. OBS do discourage the building of packages for unsupported OSes (quite reasonably), and RHEL 7/CentOS 7 are out of support at the end of June this year.

CentOS 8 Stream goes out of support at the end of May this year, so I'd also be happy to drop that. RHEL 8 isn't available on OBS due to licensing changes, so if they need new fish I'd encourage them to submit an EPEL request, build their own packages, or engage someone commercially.

openSUSE Leap 42.3 is out of support and can be dropped. Leap 15.4 still seems to be supported, although https://en.opensuse.org/Lifetime makes it sound like that's not likely to be the case in the long term. I think 15.6 packages can be installed on SLE?

Ubuntu should be fine. There's rustc 1.75 on Focal (20.04, still in support). Trusty (14.04) is already skipped and it's out of LTS this month. Xenial (16.04) and Bionic (18.04) are the only versions we'd drop and they're both out of standard support as well; if someone needs new fish on those platforms I'd probably encourage them to build their own packages or engage someone commercially. See https://wiki.ubuntu.com/Releases and https://launchpad.net/ubuntu/+source/rustc

Debian is harder. Debian 13 isn't out yet. I think the easiest thing is going to be the Ubuntu backports, as the Herculean work to move to older versions of dependencies and metadata has already been done. There's some magic to allow a bootstrapped package which I believe is acceptable to the OBS team, although it looks like the bootstrap script has been updated and the magic will need to be brought into line. I'll take a poke at that soon.

An AppImage should be easy once other platforms are working. The attempts I've made in the past use the openSUSE packages and transform that into an AppImage, and I think that's still the way to go, but with the much-reduced dependency tree and static linking of the package it should be much more straightforward to get it working.

[krobelus]

Ok that sounds promising. I do not intend to work on this, you got it.
I don't think having Debian stable PPAs (or any other PPA) lag behind would be a blocker. It will catch up.

If people are using nightly builds, it would be good to provide those on at least one platform for a while.
But I don't know how many people use them, I don't think we have a way to tell.

[krobelus]

and uploading a package containing a bunch of binaries is forbidden by policy

Doesn't that contradict our release checklist which says to upload Ubuntu and Debian packages via dput and osc commit respectively.

[zanchey]

Those are source packages only - they have the tarball, a spec file and a Debian source package (dsc).

[zanchey]

The development build pipeline has been reenabled and I've updated the release checklist to reflect the new steps. I've left the Ubuntu builds disabled until #10474 is fixed.