Build fails on multi-arch images with "docker exporter does not currently support exporting manifest lists"

[pjmpsu]

I am able to successfully build linux/arm or linux/amd64 images. When attempting to pass in multiple params to IMAGE_PLATFORM like so IMAGE_PLATFORM: linux/amd64,linux/arm64. The result is as follows:

#23 exporting to oci image format
#23 ERROR: docker exporter does not currently support exporting manifest lists
------
 > exporting to oci image format:
------
error: failed to solve: docker exporter does not currently support exporting manifest lists
FATA[0046] failed to build: build: exit status 1        
FATA[0046] failed to run task: exit status 1

[cheeseball212]

This is a great idea! Adding multi arch support would be very beneficial and streamline processes. We have employees using new mac systems with Apple's M1 processor that have reported their docker containers no longer work. We have since been looking for a way to support multiple platforms in the same image and this could certainly help in achieving that.

[joaosa]

I am also very interested in this, but it seems like an upstream issue. Look here and here

[holgerstolzenberg]

This is indeed something that becomes more and more necessary. With the emerging ARM technology introduce by Apple M1 chipset, more and more of our developers are in need of a multi-arch build for local development. Using hard coded architecture bound tags my-image:1.2.3-linux-amd64 ist not feasible, as this changes from dev env to dev env.

Currently we are using a variant of the docker-image resource (ljfranklin/docker-buildx-resource) that supports buildx, but like the docker-image resource this one should also be regarded as deprecated and everyone is encouraged to migrate to oci-build-task and registry-image resource. The resource itself is not maintained that well (no offence here).

Also - we lastly experienced some problems related to the lists.manifest.json in Artifactory using this resource, but could not clarify upon now if it is related to the resource or Artifactory (being our artifact storage system).

We really would want to migrate to oci-build-task and registry-image resource for various reasons:

• documentation
• stability
• performance
• caching
• maintenance
• being official Concourse resources

...but not being able to do multi-arch builds is a showstopper for us.

As already mentioned, the build already runs for multiple architectures, it is only a matter of the exporter to support it.

Would be great to see that coming soon. ✌️

[dackroyd]

Got further with this by running a custom build of the oci-build-task, where its --output is set to "type=oci,dest="+imagePath. The image builds successfully with buildctl, however the final steps of the oci-build-task fail because it requires a manifest.json to be present in the tarball, which the oci format does not include.

Having then bypassed that section as well, and attempting to use the resulting tarball with the registry-image-resource, the same manifest.json failure pops up. Managed to dig up an old open issue from there pointing to the problem: concourse/registry-image-resource#49 (comment)

[...] would it be accurate to say the current oci format (supported by in), while having Docker-specific manifest.json in it, is still valid OCI at least? i.e. it's a superset?

I suspect that being able to properly produce the oci output via this oci-build-task, and the registry-image-resource also supporting the oci format could resolve the issue of multi-arch support?

[dackroyd]

I believe this is resolved by #93 and concourse/registry-image-resource#321. The later appears to have made it into registry-image-resource release v1.6.0: concourse/registry-image-resource@v1.5.1...v1.6.0, while oci-build-task has not yet cut a release since the corresponding change went in.

[brunellie]

To fix this just add OUTPUT_OCI: true in the task params.

[scottwsides]

To fix this just add OUTPUT_OCI: true in the task params.

Could you be more specific? Where are these 'task params'? What does your final docker buildx build.... command look like

[meezaan]

@brunellie this worked for me.

@scottwsides see screenshot.

[kolserdav]

I use --output type=image instead of --output type=docker and it works for me:

docker buildx build --platform="linux/arm64,linux/amd64"  --output="type=image"

[brunellie]

@scottwsides This is an issue on the Concourse oci-build-task. I was mentioning the params field in the generic Concourse task configuration. I'm using the task to build multi-arch so there's no command to show