-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
scaleSetListener ignores scaling settings while acquiring the jobs #3446
Comments
Hello! Thank you for filing an issue. The maintainers will triage your issue shortly. In the meantime, please take a look at the troubleshooting guide for bug reports. If this is a feature request, please review our contribution guidelines. |
Hey @prizov, We are aware of this problem, and we are gradually working on the fix. This problem is the reason why we added capacity information in the latest release, so the server can be aware of it, and will not offer any jobs to the listener that it won't be able to handle. I will however bring this issue to the team, to check if we should do something on the listener before server changes are ready. |
Checks
Controller Version
0.9.0
Deployment Method
Helm
Checks
To Reproduce
Describe the bug
According to the official docs, the listener is supposed to check if it's able to scale up to the desired count before acquiring a job:
However, when we set
maxRunners
to0
to prevent listeners from acquiring new jobs while waiting for the completion of the jobs that are currently in progress, we noticed that the listener continues to acquire jobs despite the autoscaling settings (see logs).We have a multi-cluster setup and expected all jobs to be handled by one cluster, while another has
maxRunners
set to0
. But, this resulted in queued jobs, as a listener was still acquiring the jobs.I don't see any checks in the listener code before it acquires the jobs
actions-runner-controller/cmd/githubrunnerscalesetlistener/autoScalerService.go
Lines 100 to 205 in f7eb88c
Describe the expected behavior
The listener should verify autoscaling settings to ensure it can handle a job before acquiring it
Listener Logs
https://gist.github.com/prizov/64a1045e83cb8b1612087109602cc2c1
Note the scaling settings on line 10 https://gist.github.com/prizov/64a1045e83cb8b1612087109602cc2c1#file-gistfile1-txt-L10
The text was updated successfully, but these errors were encountered: