Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests don't run after migrating to pyscaffold #649

Open
ofosos opened this issue Jun 17, 2022 · 14 comments
Open

Tests don't run after migrating to pyscaffold #649

ofosos opened this issue Jun 17, 2022 · 14 comments
Labels
documentation Improvements, additions and corrections to documentation

Comments

@ofosos
Copy link

ofosos commented Jun 17, 2022

Description of your problem

After migrating to pyscaffold, I get some errors trying to run tox.

Please provide a minimal, self-contained, and reproducible example.

Checkout https://codeberg.org/ofosos/rotoscope version f80378ef7dbec85dd86b4427ca9e2eee5af60779

# tox

Please provide the full traceback using the --very-verbose flag.

/Users/mark/PycharmProjects/rotoscope/venv/bin/Python /Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_tox_runner.py
Testing started at 23:10 ...
.package recreate: /Users/mark/PycharmProjects/rotoscope/.tox/.package
.package installdeps: setuptools>=46.1.0, setuptools_scm[toml]>=5, wheel
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools_scm/version.py:67: UserWarning: tag 'draft1' no version found
  warnings.warn(f"tag {tag!r} no version found")
Traceback (most recent call last):
  File "venv/lib/python3.8/site-packages/tox/helper/build_requires.py", line 16, in <module>
    for_build_requires = backend.get_requires_for_build_sdist(None)
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools/build_meta.py", line 181, in get_requires_for_build_sdist
    return self._get_build_requires(config_settings, requirements=[])
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools/build_meta.py", line 159, in _get_build_requires
    self.run_setup()
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools/build_meta.py", line 174, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 13, in <module>
    setup(use_scm_version={"version_scheme": "no-guess-dev"})
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 109, in setup
    _setup_distribution = dist = klass(attrs)
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools/dist.py", line 477, in __init__
    _Distribution.__init__(
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 293, in __init__
    self.finalize_options()
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools/dist.py", line 901, in finalize_options
    ep(self)
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools/dist.py", line 922, in _finalize_setup_keywords
    ep.load()(self, ep.name, value)
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools_scm/integration.py", line 75, in version_keyword
    _assign_version(dist, config)
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools_scm/integration.py", line 48, in _assign_version
    maybe_version = _get_version(config)
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools_scm/__init__.py", line 151, in _get_version
    parsed_version = _do_parse(config)
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools_scm/__init__.py", line 98, in _do_parse
    version = _version_from_entrypoints(config) or _version_from_entrypoints(
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools_scm/_entrypoints.py", line 37, in _version_from_entrypoints
    version: Optional[ScmVersion] = _call_entrypoint_fn(root, config, ep.load())
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools_scm/_entrypoints.py", line 13, in _call_entrypoint_fn
    return fn(root, config=config)
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools_scm/git.py", line 146, in parse
    return _git_parse_inner(
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools_scm/git.py", line 181, in _git_parse_inner
    return meta(
  File "/Users/mark/PycharmProjects/rotoscope/.tox/.package/lib/python3.8/site-packages/setuptools_scm/version.py", line 195, in meta
    assert parsed_version is not None, "Can't parse version %s" % tag
AssertionError: Can't parse version draft1
ERROR: invocation failed (exit code 1), logfile: /Users/mark/PycharmProjects/rotoscope/.tox/.package/log/.package-2.log
================================== log start ===================================


An error occurred while building the project, please ensure you have the most updated version of setuptools, setuptools_scm and wheel with:
   pip install -U setuptools setuptools_scm wheel



=================================== log end ====================================
ERROR: FAIL could not package project - v = InvocationError("/Users/mark/PycharmProjects/rotoscope/.tox/.package/bin/python venv/lib/python3.8/site-packages/tox/helper/build_requires.py setuptools.build_meta '' ''", 1)

Process finished with exit code 2

Empty suite

Please provide any additional information below.

This commit of mine fixes the problem: https://codeberg.org/ofosos/rotoscope/commit/239d3a0cdb0e34bb1fb5c83878db7669e2f83905

Versions and main components

  • PyScaffold Version: 4.2.2
  • Python Version: 3.8
  • Operating system: Mac OS 12.4 / Apple Silicon
  • How did you install PyScaffold: pip
@abravalheri
Copy link
Collaborator

abravalheri commented Jun 17, 2022

Hi @ofosos, sorry for the inconvenience. Unfortunately this is something we cannot fix in Pyscaffold itself. However we can (and should) document a practice that would eliminate this error in our guides.

The gist of what is happening is the following:

PyScaffold automatically configures setuptools-scm in the generated project. This means that the version will be derived from the closes git tag which are expected to be similar to a version string (e.g. "v0.0.1.dev").

It seems, however, that setuptools-scm is stricter than we thought (I don't know if it changed since PyScaffold first adopted it).

We should modify the documentation in https://pyscaffold.org/en/stable/migration.html with steps to ensure that this error will not happen.

@abravalheri abravalheri added bug Something isn't working documentation Improvements, additions and corrections to documentation and removed bug Something isn't working labels Jun 17, 2022
@abravalheri
Copy link
Collaborator

On second thought, I might need a bit more time to investigate if this is not a problem with setuptools-scm.

@ofosos
Copy link
Author

ofosos commented Jun 18, 2022

In particular, I need to pass 'find:' instead of 'findnamespace:' to the setup.cfg. And there's also 'term-missing' in cov-report. Both are kind of weird, but the latter just looks like a bug.

@ofosos
Copy link
Author

ofosos commented Jun 18, 2022

Creating a tag 'v0.4' and running tox with the findnamespace: option results in the same output. Not working for me.

@ofosos
Copy link
Author

ofosos commented Jun 18, 2022

Strike that. If I delete my tag 'draft1' and change the source back to 'find_namespace:' it works for me.

@abravalheri
Copy link
Collaborator

abravalheri commented Jun 18, 2022

It seems like a limitation in setuptools-scm. I verified that setuptools-scm will always get the first tag and stop looking. So if you have draft1 and v0.4 in the same commit, it will still fail.
draft1 needs to be behind v0.4.

I will try to investigate a bit more about this limitation and propose an issue in the setuptools-scm track. I looks weird to me that setuptools-scm use a warning in the first time it see a tag that does not resemble a version, but then afterwards it fails anyway with an error...

@abravalheri
Copy link
Collaborator

While I verify with the seutptools-scm, I started to work in better instructions in the docs: #651.

You can see the rendered version of the docs here: https://pyscaffold--651.org.readthedocs.build/en/651/migration.html.

@abravalheri
Copy link
Collaborator

abravalheri commented Jun 18, 2022

A similar problem a non-version tag seems to have been reported in setuptools-scm: pypa/setuptools_scm#449 and pypa/setuptools_scm#235.

The second issue here (multiple tags in the same commit), also seems to be a well-known problem in setuptools-scm: pypa/setuptools_scm#521.

@abravalheri
Copy link
Collaborator

abravalheri commented Jun 18, 2022

A similar problem a non-version tag seems to have been reported in setuptools-scm: pypa/setuptools_scm#449

Indeed this seems to be a problem only for certain formats of tags:

The following test will pass:

mkdir /tmp/myproj
cd /tmp/myproj
cat << EOS > pyproject.toml
[build-system]
requires = ["setuptools", "setuptools-scm"]

[project]
name = "myproj"
dynamic = ["version"]


[tool.setuptools_scm]
EOS

touch myproj.py
git init .
git add myproj.py pyproject.toml
git commit -m "Initial Commit"
git tag "notaversion"
pipx run build  # OR pipx run --spec setuptools-scm -- python -m setuptools_scm
# ...
# Successfully built ...

But if I change the tag to "notaversion3", it will fail:

git tag -d not-a-version
git tag notaversion3
pipx run build  # OR pipx run --spec setuptools-scm -- python -m setuptools_scm
# ...
# AssertionError: Can't parse version notaversion3

@ofosos
Copy link
Author

ofosos commented Jun 18, 2022

What about the 'term-missing' in cov-report?

@abravalheri
Copy link
Collaborator

abravalheri commented Jun 18, 2022

Hi @ofosos, I tried your repository at the commit 1b6337f, and I was not able to spot anything out of the ordinary:

image

For completeness, this is what the report looks like in my machine:

image

Are you running the tests via tox?

Maybe I am not able to reproduce your results because I am running in a different OS.
Or maybe there is a problem with our configuration for coverage or pytest-cov, or with the tools themselves, but I still haven't managed to reproduce...
Can you share some quick code snippet showing the steps to achieve this outcome?

What I did was the following:

git clone https://codeberg.org/ofosos/rotoscope.git
cd rotoscope
virtualenv -p python3.8 .venv
.venv/bin/python -m pip install -U pip tox
.venv/bin/tox

(I will probably just be able to have a look on this tomorrow)

@ofosos
Copy link
Author

ofosos commented Jun 18, 2022

If you look at this commit you will see what I corrected there to get it running:
https://codeberg.org/ofosos/rotoscope/commit/041f747f4cbfd0a5f4ae732f9fa490611878595f

@abravalheri
Copy link
Collaborator

abravalheri commented Jun 19, 2022

Hi @ofosos, I still don't see anything out of ordinary when the coverage configuration uses the term-missing report setting.

I tried the following:

git clone https://codeberg.org/ofosos/rotoscope.git  # HEAD at 1b6337f
cd rotoscope
sed -i 's/--cov-report html/--cov-report term-missing/' setup.cfg
virtualenv -p python3.8 .venv
.venv/bin/python -m pip install -U pip tox
.venv/bin/tox

And the output seems to be completely fine:

image

The difference is that with the html configuration, coverage will create an html report. With the term-missing configuration, coverage will report directly in the terminal (hence term) + it will show the line numbers that the tests did not hit (hence missing). The difference between multiple reporters is detailed in this document: https://pytest-cov.readthedocs.io/en/latest/reporting.html.

(The interpretation of the report is that in line 24 of src/rotoscope/rotoscope.py was not hit by the tests).

PyScaffold adds this configuration as the default because it is the most straight forward (instant feedback) and don't require users opening a file in the browser. But the users can always edit setup.cfg with their preferences (as you did), or run tox -- --cov-report html ad-hoc, when they want to see the full HTML report.

Please let me know if you are seeing something different. Unfortunately I don't have in hand a macOS platform to try to replicate your environment, but maybe I can help. If coverage (or pytest-cov) is behaving differently depending on the OS/hardware/etc it might be the case we have to report this problem to their repositories.

@ofosos
Copy link
Author

ofosos commented Jun 19, 2022

Oh, Thanks @abravalheri , I'm sorry I wasted your time here. I'll have a look at the docs and see if I can help out there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements, additions and corrections to documentation
Projects
None yet
Development

No branches or pull requests

2 participants