-
Notifications
You must be signed in to change notification settings - Fork 133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Unify containers #953
base: master
Are you sure you want to change the base?
Conversation
The current CI error could be fixed by moving
|
Yep, I think that's showing that it's actually a good test -- it's installing an existing x86_64 package from bioconda channel into the ARM container. I think the fix is to build both amd and arm test packages on the runner (using build-env), and then get them to the create-env's Dockerfile.test. Haven't quite worked out how best to do that yet. Another big step is to get the main bioconda-utils tests to use these new containers when doing all the unit tests. Also haven't gotten there yet... |
At martin-g#8 I worked on another solution - pass The latest build fails with:
I pass |
The current issue seems to be that we need to get the images built by buildah/podman over into docker storage so they can be used by bioconda-utils to test. I'm unable to get One possible workaround might be to run a more recent version of podman. But the Ubuntu 22.04 LTS repos don't support that. So this workaround would mean building podman from source, which I think would have to happen on every build which I don't think is appropriate. Another workaround might be to push the just-built images to GitHub container registry, and then pull from there. That also has the minor benefit of being able to parallelize a little more and split the workflow into jobs. However we'd need to figure out what the storage situation is like, and if we can/should delete images immediately after testing. And of course the other option is to not test at all, but that seems a shame now that we have just about everything in place to do that. Any ideas? Pinging @mbargull for podman advice... Edit: here's the error from |
Is there a fedora/redhat image that you can use instead of ubuntu? This should maybe have a newer podman version. |
Based on GitHub runners, doesn't look like it. |
For the record, I think I'm going to take the approach of pushing to ghcr.io and then pulling back down when needed for docker. Seems to take something like ~25s total for push and pull per image. |
After some code review (thanks @mbargull) here is the plan moving forward to simplify:
|
Since the immediate focus is on ARM container support handled in a unified way, and these changes do that, I propose we start using this as-is and then separately as the next stage do the proposed simplification work. @mbargull, thoughts? |
This addresses bioconda/bioconda-containers#74 (comment) by unifying all container builds within this repo, driven by a single bash script.
build-images.yml
build.sh
script is provided for local developmentxrefs: