Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix test_regressions tests failing in GitHub Actions #159

Open
tpvasconcelos opened this issue Jan 8, 2024 · 1 comment
Open

Fix test_regressions tests failing in GitHub Actions #159

tpvasconcelos opened this issue Jan 8, 2024 · 1 comment
Labels
CI Related to CI checks good first issue Good for newcomers help wanted Extra attention is needed

Comments

@tpvasconcelos
Copy link
Owner

tpvasconcelos commented Jan 8, 2024

I had to mark the test_regressions tests with XFAIL when running the checks in CI because they are currently failing when being run in GitHub Actions. See here and here.

This is probably due to small differences in output between environments (local vs. remote macOS, Windows, and Linux).

The test, for reference:

@pytest.mark.xfail(
# TODO: Fix this (i.e. re-enable these tests)!
reason=(
"Currently breaking in CI, probably due to small "
"differences in output between environments."
),
condition=os.getenv("CI") is not None,
)

Regardless of the solution, one nice feature to implement here is the ability to upload the plot artefacts that resulted in the failed tests and expose these artefacts to the relevant PR. This discussion might be useful for this:

@tpvasconcelos tpvasconcelos added good first issue Good for newcomers CI Related to CI checks labels Jan 8, 2024
@tpvasconcelos
Copy link
Owner Author

For some reason, the tests/e2e/test_examples.py::test_examples_width_height_set[lincoln_weather-load_lincoln_weather] hangs forever and never finishes.

Some examples here: https://github.com/tpvasconcelos/ridgeplot/actions/runs/7481625128/job/20363565240

This ran succeeded but it took over 14minutes to run the e2e test suite: https://github.com/tpvasconcelos/ridgeplot/actions/runs/7481625128/job/20363565240

For this reason, I have also changed the xfail mark to a skipif (8faa9df)

@tpvasconcelos tpvasconcelos added the help wanted Extra attention is needed label Jan 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Related to CI checks good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant