How to run build steps within custom (docker) containers.

[stefanseefeld]

Hello,

I'd like to migrate some CI pipelines from jenkins to buildbot. In our original setup, we are running jenkins pipelines within custom docker containers, so I wonder whether it's possible to do the same with buildbot.
I have read about Latent Build Workers, but I don't think they meet our needs. In particular, it seems latent build workers will contain buildbot worker configurations, i.e. they run the worker themselves, while in the use-case I have in mind it's the other way around: it's the worker that runs the container. In particular, the corresponding image does not contain any buildbot infrastructure.
Is that a supported use-case ?

[ivanov17]

I think a worker can run containers if the worker is running on the host. But the worker must run as an unprivileged user. It can also execute commands in containers using docker/podman exec.

[stefanseefeld]

Yes, that's exactly what I'm looking for. I was hoping that such a facility already exists (i.e., doesn't require me to write my own worker extension), given how common a use-case this is these days (e.g. all the github workflows seem to be working like that)

In fact, I want to set up my own CI platform that can be configured just like github workflows: Some yaml configuration to control multiple builds for different platforms (linux / macos / windows), where the build steps are typically all the same, just the platform (container) setup differs.

[ivanov17]

I think you can also look at buildbot-travis. I've never used it, but it seems to do something similar. Perhaps in your case you should adapt it to use with local workers.

My buildbot setup uses latent workers to perform a number of tasks in a few specially configured containers that are chosen based on repositories, changes, etc. This is not what you are looking for 🙂