Update testing environments

[OriolAbril]

Description

The initial goal of the PR was to test if ArviZ was already compatible with numpy 2.0 (which had just been pre-released as a beta version). Several dependencies weren't compatible yet, so it wasn't feasible to do. While waiting for this, we also saw ArviZ wasn't compatible with the latest scipy pre-release #2330, and the fix wasn't released before the actual release, generating many more issues like #2336. Moreover, even though the goal is to follow SPEC 0 guidance on dependency minimum version, having numba as optional dependency in fact pushes us the other way around, it often takes months after the release of a minor python version until we can incorporate it into our CI.

To try and handle all this, this PR updates our testing environments from 4 to 5. Now the environments are:

• 3 envs: "latest" 3 pythons with all optional dependencies, we ensure all test suite is executed. "latest" won't always be true, between a python release and numba being compatible with it the latest python version won't be there.
• 1 env: latest python with no optional dependencies. now always latest. Not all test suite is executed, any test that requires a missing dependency is skipped.
• 1 env: 2nd to last python released with no optional dependencies and nightly versions for scientific python packages. This should give us a significantly larger window for changes in dependencies that break ArviZ and for some reason go unnoticed.

Checklist

• Follows official PR format
• Includes new or updated tests to cover the new feature
• Code style correct (follows pylint and black guidelines)
• Changes are listed in changelog

---

📚 Documentation preview 📚: https://arviz--2321.org.readthedocs.build/en/2321/

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.98%. Comparing base (ab82f80) to head (8e8feaf).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2321      +/-   ##
==========================================
+ Coverage   86.84%   86.98%   +0.14%     
==========================================
  Files         123      123              
  Lines       12745    12744       -1     
==========================================
+ Hits        11068    11085      +17     
+ Misses       1677     1659      -18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[OriolAbril]

Should be ready to merge!

[ahartikainen]

Sounds ok

So the current setup tests some github repo too, but we would skip these?

[ahartikainen]

Are still using this file somewhere?

[OriolAbril]

Yes, all the jobs that now have the _optional suffix install the requirements in this file. It is done in the next step now though because it doesn't happen for all jobs and so needs its own condition to be evaluated.

[OriolAbril]

So the current setup tests some github repo too, but we would skip these?

The preview job currently installs xarray from github. With this change it won't install from github but it will install nigthly versions for xarray, numpy, scipy, h5py, matplotlib... So when it comes to xarray the behaviour is quite similar, at most testing on a day old nightly instead of exact present moment github version. But for all other scientific python libraries we will catch breaks and issues way earlier than we did before.