lmfit-py is affected by the uncertainities Python3.12 incompatibility

[picca]

Hello, I am packaging larch for Debian, and while trying to run the unti test with pytest, I get this error message.

_______________ ERROR collecting tests/test_basic_processing.py ________________
ImportError while importing test module '/tmp/autopkgtest.fz045e/autopkgtest_tmp/tests/test_basic_processing.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_basic_processing.py:9: in <module>
    from utils import TestCase
tests/utils.py:13: in <module>
    from larch import Interpreter, InputText
/usr/lib/python3/dist-packages/larch/__init__.py:25: in <module>
    import lmfit
/usr/lib/python3/dist-packages/lmfit/__init__.py:38: in <module>
    from .confidence import conf_interval, conf_interval2d
/usr/lib/python3/dist-packages/lmfit/confidence.py:10: in <module>
    from .minimizer import MinimizerException
/usr/lib/python3/dist-packages/lmfit/minimizer.py:41: in <module>
    from .parameter import Parameter, Parameters
/usr/lib/python3/dist-packages/lmfit/parameter.py:10: in <module>
    from uncertainties import correlated_values, ufloat
/usr/lib/python3/dist-packages/uncertainties/__init__.py:225: in <module>
    from .core import *
/usr/lib/python3/dist-packages/uncertainties/core.py:22: in <module>
    from past.builtins import basestring
/usr/lib/python3/dist-packages/past/builtins/__init__.py:54: in <module>
    from past.builtins.misc import (apply, chr, cmp, execfile, intern, oct,
/usr/lib/python3/dist-packages/past/builtins/misc.py:45: in <module>
    from imp import reload
E   ModuleNotFoundError: No module named 'imp'
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!

It seems that uncertainties try to load the removed imp package.

The problem is that uncertainties seek a new maintainer, so I do not know if this issue will be fixed.

lmfit/uncertainties#179

let's see.

[newville]

@picca Thanks for bringing this to everyone's attention!

[newville]

@picca If I use a fresh mamba environment, it looks like uncertainties is working fine.

~> mamba create -n py312test python=3.12 numpy scipy matplotlib pip
~> mamba activate py312test
~> python -m install "lmfit[all]"
~> python
Python 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:38) [Clang 15.0.7 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import  uncertainties
>>> uncertainties.__version__
'3.1.7'
>>> from uncertainties import correlated_values, ufloat
>>> u = ufloat(3, 0.5)
>>> u**2
9.0+/-3.0

Most of the lmfit tests run for me with Python 3.12, including the ones that import and use the uncertainties package.

~> cd [lmfit_py_master]
~> cd tests
~> pytest

does give one failure (related to a scipy deprecation about maxiter that we know has been coming finally breaking), but not related to uncertainties.

Maybe this is something specific to the Debian environment?

[picca]

On Debian if I try to import imp I get this error

$ python3.12
Python 3.12.1 (main, Dec  8 2023, 16:20:54) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import imp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'imp'

Here from the changelog of Python

https://docs.python.org/3/whatsnew/3.12.html

The asynchat, asyncore, and imp modules have been removed, along with several [unittest.TestCase](https://docs.python.org/3/library/unittest.html#unittest.TestCase) [method aliases](https://docs.python.org/3/whatsnew/3.12.html#unittest-testcase-removed-aliases).

So there is at least to options :)

• Python3.12 from mamba contain an imp module
• uncertainties from mamba have been patched to support Python3.12 without imp.
• other option ?

thanks

[newville]

@picca -- OK, thanks.... Not sure what the best solution is...

[picca]

Do you know where is the code of uncertainties on mamba ?

I should investigate also the futur package... the past modules if from python-future...

[newville]

@picca In the mamba environment that I installed, uncertainties 3.1.7 was installed by pip.

The past module there is version 0.18.3. That does use importlib in place of imp (though it does use this in a try-except that might use from imp import ...).

If I can hazard a guess, it may be that your Debian environment is at /usr/lib/python3.12/ but (because of otherwise "very bad" --- oh sorry, I meant to say "very reasonable" -- packaging decisions by Debian), this can also import modules from the generic non-versioned, non-environment-specific /usr/lib/python3/dist-packages/, which is where your code is finding the past module. Perhaps (or "likely"), that is out-dated for Python 3.12.

[picca]

I just found this also :)). I think that they are very conservative and try to avoid breaking packages. Now there is this Python3.12 migration it it breaks lot's of packages. So thi sis the time for investigation. It is a never ending story.

Thansk a lot, I will try to push for a new python-future version in Debian.

Fred

[picca]

nevertheless uncertainties is still not maintained.. so this is quite fragile ;)

[newville]

@picca I agree with the concerns about the maintenance of uncertainties and hope we can remedy that.