Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] #1217 Added feature to emulate gitlab hosted runner using the --container-emulate option with the name of one of the gitlab preset. #1219

Merged
merged 6 commits into from
May 16, 2024

Conversation

nicDamours
Copy link
Contributor

@nicDamours nicDamours commented May 14, 2024

summary

Added an option to prefill memory, kernel-memory and cpus docker options based on the configuration of Gitlab's hosted runner. This is useful when attempting to debug performance issue. By default, no limit is applied, the host resources are all available.

When passing the option --container-emulate= with the name of a gitlab hosted runner, the docker resources is limited to the resource available for this class of hosted runner. As of this PR, those are the values

name memory CPU
saas-linux-small 8 GB 2
saas-linux-medium 16 GB 4
saas-linux-large 32 GB 8
saas-linux-xlarge 64 GB 16
saas-linux-2xlarge 128 GB 32

see the gitlab documentation here

Of course, you are also limited to the host resources. You cannot have 128 GB of RAM on a host that only has 16 GB.

As of right now, I've only added support for the base linux machine. This does not include runner with a GPUs. This could be a nice amelioration in the futur. The runners names does not contains the architecture, as this is based on the host CPU, and cannot be emulated.

Usage

You can pass an additional option --containter-emulate with one of the previously mentionned runner name. For example, using the name saas-linux-small will limit the container resource to 8 GB or RAM and 2 CPUs.

gitlab-ci-local --cwd /path/to/.gitlab.yml --container-emulate=saas-linux-small

You can check the memory limit and CPU usage using the docker stats {container_id} command.

image

This PR Closes #1217

Added feature to emulate gitlab hosted runner using the `--emulate` option with the name of one of the gitlab preset.
src/gitlab-preset.ts Outdated Show resolved Hide resolved
src/gitlab-preset.ts Outdated Show resolved Hide resolved
Changed initBashSpy function to return the spy in order to validate function call on it.
Changed tests to use initBashSpy function.
Changed memory preset to use `* 1024`, which helps readability.
src/index.ts Outdated Show resolved Hide resolved
@ANGkeith ANGkeith changed the title [FEATURE] #1217 Added feature to emulate gitlab hosted runner using the --emulate option with the name of one of the gitlab preset. [FEATURE] #1217 Added feature to emulate gitlab hosted runner using the --container-emulate option with the name of one of the gitlab preset. May 15, 2024
Changed emulation option from `--emulate` to `--container-emulate` to follow container option convention.
src/index.ts Outdated Show resolved Hide resolved
@ANGkeith
Copy link
Collaborator

thanks for the contribution !

src/argv.ts Outdated Show resolved Hide resolved
Added gitlab hosted runner documentation url in the description of the new `--container-emulate` option.
Replaced "container-emulate" with "containerEmulate".
@firecow
Copy link
Owner

firecow commented May 15, 2024

@nicDamours I don't see the purpose of the amd64 postfix.

@nicDamours
Copy link
Contributor Author

@nicDamours I don't see the purpose of the amd64 postfix.

@firecow it's the name given by gitlab for those specific runner: https://docs.gitlab.com/ee/ci/runners/hosted_runners/linux.html#machine-types-available-for-linux---x86-64

It's not essential for this feature but it helps linking gitlab's runner to the preset defined here.

@firecow
Copy link
Owner

firecow commented May 16, 2024

@nicDamours I don't see the purpose of the amd64 postfix.

@firecow it's the name given by gitlab for those specific runner: https://docs.gitlab.com/ee/ci/runners/hosted_runners/linux.html#machine-types-available-for-linux---x86-64

It's not essential for this feature but it helps linking gitlab's runner to the preset defined here.

I can imagine people with ARM processors will be very confused, when they realise that using such a preset, actually works 😄

@nicDamours
Copy link
Contributor Author

I can see how that would be confusing. I will remove the amd64 prefix 🙂

Removed amd64 suffix from possible runner emulation name.
src/gitlab-preset.ts Outdated Show resolved Hide resolved
Removed "AMD_64" from runner preset enum.
@firecow firecow merged commit e9db7b8 into firecow:master May 16, 2024
9 checks passed
@nicDamours nicDamours deleted the feature/#1217/add-runner-preset branch May 16, 2024 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for predefined runner instance class
3 participants