Skip to content
This repository has been archived by the owner on Mar 14, 2023. It is now read-only.

What if containers take more than 30 secs to start? #39

Open
itsmesuniljacob opened this issue Feb 9, 2021 · 2 comments
Open

What if containers take more than 30 secs to start? #39

itsmesuniljacob opened this issue Feb 9, 2021 · 2 comments

Comments

@itsmesuniljacob
Copy link

Hi Team,

There is scenario where many of our containers may take more than 30 secs to start. In other words, 30 seconds (probably) will not be enough for new replicas to start when VMs receive preemption signal.

Is it possible to modify this draining_timeout_when_node_expired_ms values to 45 secs to solve the above problem?

@nrx-ops
Copy link

nrx-ops commented Feb 11, 2021

Hi,
I have same issue.
According to documentation : https://cloud.google.com/compute/docs/instances/preemptible#preemption-process
Compute Engine sends a preemption notice to the instance in the form of an ACPI G2 Soft Off signal. You can use a shutdown script to handle the preemption notice and complete cleanup actions before the instance stops.
If the instance does not stop after 30 seconds, Compute Engine sends an ACPI G3 Mechanical Off signal to the operating system.

I think there is no way to override this 30s deadline after G2 ACPI call.

I reduce downtime with using replica and pod anti-affinity.

@itsmesuniljacob
Copy link
Author

Sure

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants