Set "zfs=" or "root=zfs=" or "root=ZFS=" depending on system initramfs build tool or make it user configurable instead of basing it on distro

[sotiris-bos]

ZFSBootMenu build source

Local build, dracut

ZFSBootMenu version

2.2.2

Boot environment distribution

Arch Linux

Problem description

I am using dracut as my initramfs build tool on Arch instead of mkinitcpio for my system images on all my computers.

zfsbootmenu-core.sh at line 977 sets the "zfs=" or "rootfs=" kernel parameters based on the distribution. In my case, it was setting it to "zfs=zroot/enc/ROOT" instead of "root=ZFS=zroot/enc/ROOT" since I am on Arch.

This seems to somehow work on my desktop but it failed to work on my server.
The server is running:

• Arch 5.15.82-1-lts
• dracut 056
• zfs-linux-lts 2.1.7_5.15.82.1-1
• zfsbootmenu 2.2.2-1

I wanted to switch from GRUB to ZBM before updating because updating broke GRUB on my desktop and I had to fix it on an unbootable system.

It seems like dracut-created initramfs images expect "root=" to be set in order for sysroot.mount to be properly set by zfs-generator and in my case it was not being set correctly. That meant dracut-mount was falling back to the mount-zfs.sh script. This in turn must have caused a race condition between dracut-mount and zfs-import-cache.service, with zfs-import-cache.service importing the root pool and dracut-mount trying to import it again via mount-zfs.sh and failing with:

server dracut-mount[5226]: cannot import 'zroot': a pool with that name already exists
server dracut-mount[5226]: use the form 'zpool import <pool | id> <newpool>' to give it a new name

This condition should maybe be filed as a bug on the OpenZFS repo.

After much troubleshooting, I finally edited zfsbootmenu-core.sh manually and changed "zfs=" to "root=ZFS=" on line 984, run generate-zbm again and booted to a perfectly working system. This problem might be attributed to the old kernel and tools versions installed on the server since it did not happen on my updated desktop system.

I think having an option to manually set the relevant commandline parameter would be very helpful for troubleshooting and general usage.

Here are the relevant entries from /run/initramfs/rdsosreport.txt when boot was failing:

[    4.054743] server zfs-generator: starting
[    4.057671] server systemd[405]: Successfully forked off '(direxec)' as PID 419.
[    4.068574] server systemd[405]: Successfully forked off '(direxec)' as PID 427.
[    4.072066] server systemd-fstab-generator[427]: Could not find a root= entry on the kernel command line.
[    4.074094] server systemd[405]: Successfully forked off '(direxec)' as PID 432.
[    4.077748] server systemd-gpt-auto-generator[432]: Reading EFI variable /sys/firmware/efi/efivars/LoaderDevicePartUUID-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f.
[    4.077770] server systemd-gpt-auto-generator[432]: open("/sys/firmware/efi/efivars/LoaderDevicePartUUID-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f") failed: No such file or directory
[    4.077776] server systemd-gpt-auto-generator[432]: EFI loader partition unknown, exiting.
[    4.077779] server systemd-gpt-auto-generator[432]: (The boot loader did not set EFI variable LoaderDevicePartUUID.)
[    4.084708] server systemd-fstab-generator[427]: Parsing /etc/fstab...
[    4.087978] server zfs-generator: writing extension for sysroot.mount to /run/systemd/generator/sysroot.mount.d/zfs-enhancement.conf
[    4.090174] server systemd[405]: /usr/lib/systemd/system-generators/dracut-rootfs-generator succeeded.
[    4.091260] server zfs-generator: finished
[    4.101483] server systemd-fstab-generator[427]: Parsing /sysroot/etc/fstab...
[    4.113548] server systemd[405]: /usr/lib/systemd/system-generators/systemd-debug-generator succeeded.
[    4.179121] server systemd[405]: /usr/lib/systemd/system-generators/systemd-gpt-auto-generator succeeded.
[    4.192654] server systemd[405]: /usr/lib/systemd/system-generators/systemd-fstab-generator succeeded.
[    4.199664] server systemd[405]: /usr/lib/systemd/system-generators/dracut-zfs-generator succeeded.
[   47.686113] server systemd[1]: Starting dracut mount hook...
[   47.686649] server systemd[5068]: dracut-mount.service: Executing: /bin/dracut-mount
[   47.677617] server systemd-journald[452]: Successfully sent stream file descriptor to service manager.
[   47.677692] server systemd-journald[452]: Successfully sent stream file descriptor to service manager.
[   47.709095] server dracut-mount[5068]: ZFS: No sysroot.mount exists or zfs-generator did not extend it.
[   47.709095] server dracut-mount[5068]: ZFS: Mounting root with the traditional mount-zfs.sh instead.
[   47.710476] server systemd[1]: dracut-pre-pivot.service: starting held back, waiting for: dracut-mount.service
[   47.710528] server systemd[1]: initrd-cleanup.service: starting held back, waiting for: dracut-pre-pivot.service
[   47.710613] server systemd[1]: systemd-journald.service: Got notification message from PID 452 (FDSTORE=1)
[   47.710644] server systemd[1]: systemd-journald.service: Added fd 43 (n/a) to fd store.
[   47.710695] server systemd[1]: systemd-journald.service: Got notification message from PID 452 (FDSTORE=1)
[   47.710722] server systemd[1]: systemd-journald.service: Added fd 48 (n/a) to fd store.
[   47.716292] server systemd-udevd[581]: Received udev control message (PING)
[   47.762805] server dracut-mount[5068]: ZFS: Using zroot/enc/ROOT as root.
[   47.767096] server dracut-mount[5068]: ZFS: Importing pool zroot...
[   47.870810] server systemd-networkd[770]: Got inotify event on bus bus-api-network.
[   47.871112] server systemd-networkd[770]: sd-bus: starting bus bus-api-network by connecting to /run/dbus/system_bus_socket...
[   47.871144] server systemd-networkd[770]: Added inotify watch for /run on bus bus-api-network: 2
[   47.871171] server systemd-networkd[770]: Added inotify watch for /run/dbus on bus bus-api-network: -1
[   47.871195] server systemd-networkd[770]: sd-bus: starting bus bus-api-network by connecting to /run/dbus/system_bus_socket...
[   47.958189] server dracut-mount[5090]: cannot import 'zroot': a pool with that name already exists
[   47.958189] server dracut-mount[5090]: use the form 'zpool import <pool | id> <newpool>' to give it a new name
[   47.981844] server dracut-mount[5068]: Warning: ZFS: Unable to import pool zroot
[   47.989600] server systemd-udevd[581]: Received udev control message (PING)
[   47.989651] server dracut-mount[5068]: ZFS: No sysroot.mount exists or zfs-generator did not extend it.
[   47.989651] server dracut-mount[5068]: ZFS: Mounting root with the traditional mount-zfs.sh instead.
[   48.021255] server dracut-mount[5068]: ZFS: Using zroot/enc/ROOT as root.
[   48.025101] server dracut-mount[5068]: ZFS: Importing pool zroot...
[   48.121592] server dracut-mount[5141]: cannot import 'zroot': a pool with that name already exists
[   48.121592] server dracut-mount[5141]: use the form 'zpool import <pool | id> <newpool>' to give it a new name
[   48.144991] server dracut-mount[5068]: Warning: ZFS: Unable to import pool zroot
[   48.152702] server systemd-udevd[581]: Received udev control message (PING)
[   48.152755] server dracut-mount[5068]: ZFS: No sysroot.mount exists or zfs-generator did not extend it.
[   48.152755] server dracut-mount[5068]: ZFS: Mounting root with the traditional mount-zfs.sh instead.
[   48.161309] server dracut-mount[5068]: ZFS: Using zroot/enc/ROOT as root.
[   48.165711] server dracut-mount[5068]: ZFS: Importing pool zroot...
[   48.262386] server dracut-mount[5184]: cannot import 'zroot': a pool with that name already exists
[   48.262386] server dracut-mount[5184]: use the form 'zpool import <pool | id> <newpool>' to give it a new name
[   48.285218] server dracut-mount[5068]: Warning: ZFS: Unable to import pool zroot
[   48.292686] server systemd-udevd[581]: Received udev control message (PING)
[   48.292743] server dracut-mount[5068]: ZFS: No sysroot.mount exists or zfs-generator did not extend it.
[   48.292743] server dracut-mount[5068]: ZFS: Mounting root with the traditional mount-zfs.sh instead.
[   48.321251] server dracut-mount[5068]: ZFS: Using zroot/enc/ROOT as root.
[   48.325133] server dracut-mount[5068]: ZFS: Importing pool zroot...
[   48.421297] server dracut-mount[5226]: cannot import 'zroot': a pool with that name already exists
[   48.421297] server dracut-mount[5226]: use the form 'zpool import <pool | id> <newpool>' to give it a new name
[   48.444004] server dracut-mount[5068]: Warning: ZFS: Unable to import pool zroot
[   48.451742] server systemd-udevd[581]: Received udev control message (PING)
[   48.451789] server dracut-mount[5068]: ZFS: No sysroot.mount exists or zfs-generator did not extend it.

Steps to reproduce

Build zfsbootmenu on Arch with kernel 5.15.82-1-lts, dracut 056, zfs-linux-lts 2.1.7_5.15.82.1-1, zfsbootmenu 2.2.2-1.

Build system initramfs with dracut instead of mkinitcpio. Add debug to org.zfsbootmenu:commandline.

Watch boot fail with:

server dracut-mount[5068]: ZFS: No sysroot.mount exists or zfs-generator did not extend it.
server dracut-mount[5068]: ZFS: Mounting root with the traditional mount-zfs.sh instead.
server dracut-mount[5090]: cannot import 'zroot': a pool with that name already exists
server dracut-mount[5090]: use the form 'zpool import <pool | id> <newpool>' to give it a new name
server dracut-mount[5068]: Warning: ZFS: Unable to import pool zroot

[classabbyamp]

this is already user-overridable with the zfs property org.zfsbootmenu:rootprefix

[sotiris-bos]

this is already user-overridable with the zfs property org.zfsbootmenu:rootprefix

Sorry, I did not see the relevant entry in the documentation, I am closing the issue. I have seen other reports with the same problem, hopefully the keywords in this issue might be helpful.

Edit: is there a way to confirm the rootprefix used in the zfsbootmenu initramfs after setting the relevant zfs property?

[classabbyamp]

by attempting to boot, I guess