RFE: formalize Containerfile workflow for custom stage outputs

[owtaylor]

See also: containers/podman#12605

In some cases, we want a Containerfile to take control of generating layer contents and metadata in very specific ways. Examples:

• bootc images - the tooling wants to divide up layers to improve the efficiency of the update process
• Flatpaks - the tooling wants to add labels with generated metadata like the base64-encoded icon

This is somewhat possible currently, you can do this by mounting the context directory (or a subdirectory) read-write, writing a container image there, then making the next stage inherit that with a containers-transports url - FROM oci-archive:out.ociarchive.

• Working bootc example
• Working flatpak example

But there are some downsides:

• The FROM oci-archive:out.ociarchive is not context-relative, it's just a filesystem path interpreted relative to the working directory of buildah. This is inconsistent: in all other cases (COPY, run --mount, etc.) referenced filesystem paths are context-relative and are restricted to being a subdirectory of the context dir, and also creates a problem that the Containefile no longer is independent of the context dir specified on the buildah build line - a Containerfile might only work with a context-dir of ., for example.
• Because buildah doesn't really understand what is going on, it considers the stage outputing the oci archive to be unused. It's necessary to either use --skip-unused-stages=false or make a dummy reference to the stage (see the bootc Containerfile above for an example.)
• Having to use the context dir as scratch space isn't ideal - it might be desirable to to have the context dir read-only, or it might be on a filesystem with limited space.

Ideally, it would be possible to do something like:

FROM --container=from=export,type=oci-archive,src=/tmp/out.ociarchive

Though just fixing the path issue would already be an improvement.

Possible Questions

Couldn't this just be scripted using individual buildah commands? Why is a Containerfile necessary?

One place where we want to use this is as part of the Konflux build service. In this case, it's very desirable to separate out the specification of what we build (via a Containerfile) from the mechanics of how we build it (with different resource limits, on multiple architectures, etc.) Having to customize the build task mixes the two up creating an explosion of different possible tasks. It's also quite desirable to be able to just give a developer a Containerfile and say "this is how you build a Flatpak", compared to a (possibly complex) shell script.

What about compatibility with other Containerfile consumers

All I can say here is that while FROM oci-archive:<path> isn't a syntax extension to the Containerfile spec, it's still a Buildah specific extension.