Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests are flaky on low-power devices and CPU-limited containers #63

Open
enter-github-username opened this issue May 1, 2023 · 1 comment

Comments

@enter-github-username
Copy link

enter-github-username commented May 1, 2023

Some tests, like test/facility.sh, use hard-coded sleep statements to ensure that prior operations finish.

While running the test suite of sysklogd inside a heavily CPU-limited podman container, a few tests (test/memleak.sh test/facility.sh test/secure.sh test/remote.sh) showed flaky behaivior, i.e. they failed occasionally. The same behaivior showed on an old notebook with a > 10 year old i3 CPU running Debian 11. This may be a minor problem for CI environments, which often heavily restrict the available CPU time.

A simple solution would be to just sleep longer - however, this would increase the test suite runtime significantly, and I would assume that only a very small amount of users are affected by the flakyness in the first place. It may be a better idea to just add it to the documentation as a caveat.

Alternatively, it may be a good idea to, in case of an unexpected failure like an empty grep result, recheck and sleep a few times inside the tests and only fail after n retries.

@troglobit
Copy link
Owner

Yes, they are flaky. I have not had the time to fix that, use them mainly for regression testing when modifying the code base. What are you using them for?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants