Report coverage diffs in PRs or dashboard builds

[jhlegarreta]

Description

Code coverage reached a peak value around 90.11% around early Oct, 2023. Later that month, coverage dropped to 89.39% from one day to another, and it stabilized around 89.59% after some build site changes (including GNU developer tool changes or relevant flags). It has been in the 89.57%-89.59% bracket since then.

On Feb 28, 2024 coverage dropped to 89.35%, and stayed at 89.34% on Feb 29, 2024.

ITK has currently no coverage diff reports per PR, and no coverage diff reports across dates in the dashboard.

A diff report telling the coverage drop of a set of changes in a PR would be a way to identify early such changes in coverage, and possibly requesting to add a test case in the PR before merging it.

If no such service can be put in place in GitHub, a custom/in-house script that reads the coverage reports posted to the CDash dashboard and compares them to the immediately previous reports would allow identifying the files and lines whose coverage has changed, possibly posting those to a dedicated website.

Expected behavior

In order to keep a healthy code base, identifying early any coverage drops, and amending them is essential.

Actual behavior

ITK has currently no coverage diff reports per PR, and no coverage diff reports across dates in the dashboard. The only way to identify changes in coverage is to inspect one by one all files (+2900) listed in the CDash coverage build and compare them between consecutive days. This is time-consuming and impractical/unfeasible to do manually.

Reproducibility

Not applicable.

Versions

Not applicable.

Additional Information

[blowekamp]

There is currently a failing test due to time out issue. This may be contributing to the reduction in coverage.

Perhaps the failing test should be modified to be more time efficient.
https://open.cdash.org/viewTest.php?onlyfailed&buildid=9419378