Not finding bootable kernels (beginner question)

[fubar-1]

Hi, new ZBM user here, I hope this isn't a dumb question-

I have a Ubuntu 23.10 system that was installed using the installer but with the ZFS option selected. So this system boots via UEFI and has the typical default Debian bpool/rpool filesystems. Relevant hardware is an MSI MAG Z690 Tomahawk motherboard with several NVME drives installed, one of which holds the UEFI bootable partition & zfs filesystems. It boots fine, I'm using it to write this.

My plan is (was?) to fresh-install Ubuntu 23.04 onto another NVME drive, this time using ZfsBootMenu and following the supplied Debian/Ubuntu instructions for doing so.

The problem is, I created a zbm bootable USB flash drive from the prebuilt image and booted to it but it doesn't find the installed & bootable ZFS filesystem. Instead it drops to the recovery shell, where running 'zpool list' and 'zfs list' show the expected zfs pools & filesystems imported correctly. I see bpool/BOOT, etc but want to be careful so I haven't tried to manually mount it yet. zreport seems ok (?). and 'logs' displays an empty log file.

I assume a bootable kernel isn't being found yet this system boots fine. Am I doing something wrong?

A second question is (there might be a better place to ask this), if I do install a second OS onto a second drive that also boots from ZFS, will I run into trouble trying to have more than one actively bootable ZFS filesystem? I.e. will ZFS try to overlay-mount both into it's namespace and cause bad things to happen?

[classabbyamp]

ZBM does not support a separate bootpool, only a /boot directory on the rootpool

https://zfsboot.menu/en/v2.2.x/guides/general/bootenvs-and-you.html

[fubar-1]

Thanks for responding. This answers my question. This debian separated bpool / rpool situation is exactly the mess I'm trying to dig out of.

What would be the recommended approach to achieving the migration outlined above? I hesitate to make the old OS unbootable prior to having a stable new OS installed. But if that's the only path, an outline of the steps to take would really help me gain the confidence to move forward.

Debian-based distros are quite popular; it might be useful to work with me sort this out and then document it in the guide so others have a clear path to resolving the problem.

[classabbyamp]

copy your /boot to /boot.bak, unmount the /boot partition/dataset, then copy /boot.bak back to /boot, which should now be under the / mountpoint

[fubar-1]

wouldn't this result in two /boot filesystems from two datasets overlaying each other?
in any case I don't think this would work here, on this system copying /boot to /boot.bak would copy files from dataset bpool/BOOT/ubuntu_hpuj4i to rpool/ROOT/ubuntu_hpuj4i. I don't think grub can boot zfs with the new zfs feature flags on rpool (?). Isn't that the whole reason for bpool, to be grub compatible?
Sorry for the zfs baby-questions.

[ahesford]

If you have multiple installations, you should set mountpoint=/ and canmount=noauto on all roots so that only the one mounted by your initramfs will be mounted. If there are other filesystems you mount that should not be mounted on both, you'll need to set mountpoint=legacy on those and manage them with /etc/fstab.

[fubar-1]

As a fairly inexperienced zfs user, the validity of my opinion is questionable, but this 'overlay mount everything mountable' design seems a bit problematic. I think this is probably what blocked an incompatible kernel / zfs out-of-tree failed upgrade recovery effort I had to deal with a few years ago.

Would it be worth filing a enhancement request against the main zfs repo? I could be wrong but it seems like it could be easily resolved by adding an attribute at the pool or filesystem level.

[classabbyamp]

by copying then unmounting the bootpool dataset, you will not be rendering the system unbootable the old way until you zpool destroy the bootpool

[fubar-1]

I deleted the last line of my comment, it was confusing the issue. Let's focus on making the current OS visible to zbm in the thread above and focus on the multiple installation problem here.

Apologies, I'm trying to understand this and not ask stupid questions. But... By setting all root filesystems canmount=noauto, will zbm be able to mount and boot it? I'm trying to figure out how the associations work. Doesn't the OS do a 'zfs mount -a' as it comes up?

[ahesford]

You're misusing the term "overlay", and the mount semantics you talk about aren't particular to ZFS, but are general concepts of UNIX filesystems. The only thing ZFS changes is that it will try to mount every ZFS filesystem marked canmount=auto, which is why you change that property on your boot environments to avoid shadowing (not "overlaying") one with another.

I doubt the mounting behavior has anything you do with your upgrade problems,

[fubar-1]

Understood loud and clear.
I'll need to spin up a vm and test this all out to gain the confidence I need to proceed.
Sorry to have bothered you.