Coverage 6.0.1 reports less coverage than 5.5 for submodule · Issue #1232 · nedbat/coveragepy (original) (raw)

Describe the bug

pytest -k entry_dedupe --cov=reader.plugins.entry_dedupe reports:

100% coverage for entry_dedupe.py on Coverage 5.5
84% coverage for entry_dedupe.py on Coverage 6.0.1

pytest -k entry_dedupe --cov=reader reports 100% coverage for entry_dedupe.py regardless of version.

FWIW, the 6.0.1 HTML 84% report shows all module-level statements (imports, defs, assignments) as missing/red, but the function bodies as covered/green.

To Reproduce
How can we reproduce the problem? Please be specific. Don't just link to a failing CI job. Answer the questions below:

What version of Python are you using?
CPython 3.8.1
What version of coverage.py are you using? The output of coverage debug sys is helpful.
6.0.1; coverage debug sys output:

-- sys -------------------------------------------------------  
            coverage_version: 6.0.1  
                coverage_module: /Users/lemon/code/reader/.venv/lib/python3.8/site-packages/coverage/__init__.py  
                        tracer: -none-  
                        CTracer: available  
        plugins.file_tracers: -none-  
            plugins.configurers: -none-  
    plugins.context_switchers: -none-  
            configs_attempted: .coveragerc  
                                setup.cfg  
                configs_read: /Users/lemon/code/reader/setup.cfg  
                    config_file: /Users/lemon/code/reader/setup.cfg  
                config_contents: b'[metadata]\nname = reader\nversion = attr: reader.__version__\nauthor = lemon24\ndescription = A Python feed reader library.\nlong_description = file: README.rst\nlong_description_content_type = text/x-rst\nurl = https://github.com/lemon24/reader\nlicense = BSD-3-Clause\nproject_urls =\n    Documentation = https://reader.readthedocs.io/\n    Code = https://github.com/lemon24/reader\n    Issue tracker = https://github.com/lemon24/reader/issues\nclassifiers =\n    Development Status :: 5 - Production/Stable\n    Environment :: Console\n    Environment :: Web Environment\n    Intended Audience :: Developers\n    Intended Audience :: End Users/Desktop\n    License :: OSI Approved :: BSD License\n    Operating System :: OS Independent\n    Programming Language :: Python\n    Programming Language :: Python :: 3\n    Programming Language :: Python :: 3 :: Only\n    Programming Language :: Python :: 3.7\n    Programming Language :: Python :: 3.8\n    Programming Language :: Python :: 3.9\n    Programming Language :: Python :: Implementation :: CPython\n    Programming Language :: Python :: Implementation :: PyPy\n    Topic :: Internet\n    Topic :: Internet :: WWW/HTTP :: Dynamic Content :: News/Diary\n    Topic :: Internet :: WWW/HTTP :: Indexing/Search\n    Topic :: Internet :: WWW/HTTP :: WSGI :: Application\n    Topic :: Software Development :: Libraries\n    Topic :: Utilities\n    Typing :: Typed\n\nkeywords = atom, cdf, feed, rdf, rss, json feed, web feed, podcast, feed reader, feed aggregator\n\n[options]\npackages = find:\npackage_dir = = src\ninclude_package_data = true\npython_requires = >=3.7\n\ninstall_requires =\n    typing-extensions\n    feedparser>=6\n    requests>=2.18\n    # for JSON Feed date parsing\n    iso8601\n\n[options.packages.find]\nwhere = src\n\n[options.extras_require]\n\n# STABLE\n\nsearch =\n    beautifulsoup4>=4.5\n\n# UNSTABLE\n\ncli =\n    click>=7\n    # for config\n    PyYAML\napp =\n    flask>=0.10\n    humanize\n    # for config\n    PyYAML\n\n# UNSTABLE PLUGINS\n\n# mushed together for convenience\nunstable-plugins =\n\n    # enclosure-tags\n    requests\n    mutagen\n\n    # preview-feed-list\n    requests\n    beautifulsoup4\n    blinker>=1.4\n\n    # sqlite-releases\n    beautifulsoup4\n\n# DEVELOPMENT\n\n# run tests under one interpreter\ntests =\n    pytest>=4\n    pytest-randomly\n    flaky\n    hypothesis>=5\n    coverage\n    pytest-cov\n    requests-mock\n    # mechanicalsoup hard-depends on lxml (see below)\n    mechanicalsoup; implementation_name != "pypy"\n    requests-wsgi-adapter\n    # we want to test search with all known bs4 parsers.\n    # lxml (sometimes) fails to build on pypy, we don\'t bother.\n    lxml; implementation_name != "pypy"\n    html5lib\n    # for _http_utils tests\n    werkzeug\n    # mypy does not work on pypy (yet).\n    mypy; implementation_name != "pypy"\n    types-requests\n\n# build docs\ndocs =\n    sphinx\n    sphinx_rtd_theme\n    click>=7\n    sphinx-click\n    sphinx-issues\n    sphinx-hoverxref\n    sphinxcontrib-log-cabinet\n    # https://github.com/readthedocs/sphinx_rtd_theme/issues/1112\n    # https://github.com/readthedocs/readthedocs.org/issues/7858\n    docutils==0.16\n\n# other things needed to develop / test locally / make releases\ndev =\n    tox\n    pre-commit\n    build\n    twine\n\n\n[tool:pytest]\nmarkers =\n    slow: mark a test as slow.\naddopts = --no-success-flaky-report\nfilterwarnings =\n    ignore:No parser was explicitly specified::reader._search\n\n\n[coverage:run]\nbranch = true\nsource =\n    reader\n    tests\n\n[coverage:paths]\nsource =\n    src\n    */site-packages\n\n\n[flake8]\n# B = bugbear\n# E = pycodestyle errors\n# F = flake8 pyflakes\n# W = pycodestyle warnings\n# B9 = bugbear opinions\nselect = B, E, F, W, B9\nignore =\n    # slice notation whitespace, invalid\n    E203\n    # line length, handled by bugbear B950\n    E501\n    # bugbear line length; too sensitive, triggered for comments\n    # and docstrings (and adding "noqa" in comments is making things worse);\n    # black taking care of line length for code should be good enough;\n    # if enabled, we should set max-line-length = 80 (so up to 88 are allowed)\n    B950\n    # bare except, handled by bugbear B001\n    E722\n    # bin op line break, invalid\n    W503\n    # mistakenly triggered for @overload;\n    # https://github.com/PyCQA/pyflakes/issues/320 says it should be fixed\n    # in 2.1.1, but it doesn\'t seem to be (it\'s fixed in master though)\n    F811\nper-file-ignores =\n    # __init__ modules export names\n    **/__init__.py: F401\nexclude = tests/*, docs/*, scripts/*, src/reader/_vendor/*\n\n\n[mypy]\n# empty section required in 0.800; https://github.com/python/mypy/issues/9940\n[mypy-reader.__main__]\nignore_errors = True\n[mypy-reader._cli]\nignore_errors = True\n[mypy-reader._config]\nignore_errors = True\n[mypy-reader._app.*]\nignore_errors = True\n[mypy-reader._plugins.*]\nignore_errors = True\n[mypy-reader.plugins.*]\nignore_errors = True\n'  
                    data_file: -none-  
                        python: 3.8.1 (v3.8.1:1b293b6006, Dec 18 2019, 14:08:53) [Clang 6.0 (clang-600.0.57)]  
                    platform: macOS-10.15.7-x86_64-i386-64bit  
                implementation: CPython  
                    executable: /Users/lemon/code/reader/.venv/bin/python3.8  
                def_encoding: utf-8  
                    fs_encoding: utf-8  
                            pid: 7827  
                            cwd: /Users/lemon/code/reader  
                        path: /Users/lemon/code/reader/.venv/bin  
                                /Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip  
                                /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8  
                                /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload  
                                /Users/lemon/code/reader/.venv/lib/python3.8/site-packages  
                                /Users/lemon/code/reader/src  
                    environment: HOME = /Users/lemon  
                                PYTEST_ADDOPTS = -p no:randomly  
                                __PYVENV_LAUNCHER__ = /Users/lemon/code/reader/.venv/bin/python3.8  
                command_line: /Users/lemon/code/reader/.venv/bin/coverage debug sys  
                sqlite3_version: 2.6.0  
        sqlite3_sqlite_version: 3.28.0  
            sqlite3_temp_store: 0  
        sqlite3_compile_options: COMPILER=clang-6.0.0; ENABLE_FTS3_PARENTHESIS; ENABLE_FTS4  
                                ENABLE_FTS5; ENABLE_JSON1; ENABLE_RTREE  
                                THREADSAFE=1

What versions of what packages do you have installed? The output of pip freeze is helpful.
pip freeze output:

alabaster==0.7.12  
attrs==21.2.0  
Babel==2.9.1  
backports.entry-points-selectable==1.1.0  
beautifulsoup4==4.10.0  
bleach==4.1.0  
blinker==1.4  
build==0.7.0  
certifi==2021.5.30  
cfgv==3.3.1  
charset-normalizer==2.0.6  
click==8.0.1  
colorama==0.4.4  
coverage==6.0.1  
distlib==0.3.3  
docutils==0.16  
feedparser==6.0.8  
filelock==3.3.0  
flaky==3.7.0  
Flask==2.0.2  
html5lib==1.1  
humanize==3.12.0  
hypothesis==6.23.1  
identify==2.3.0  
idna==3.2  
imagesize==1.2.0  
importlib-metadata==4.8.1  
iniconfig==1.1.1  
iso8601==0.1.16  
itsdangerous==2.0.1  
Jinja2==3.0.2  
keyring==23.2.1  
lxml==4.6.3  
MarkupSafe==2.0.1  
MechanicalSoup==1.1.0  
mutagen==1.45.1  
mypy==0.910  
mypy-extensions==0.4.3  
nodeenv==1.6.0  
packaging==21.0  
pep517==0.11.0  
pkginfo==1.7.1  
platformdirs==2.4.0  
pluggy==1.0.0  
pre-commit==2.15.0  
py==1.10.0  
Pygments==2.10.0  
pyparsing==2.4.7  
pytest==6.2.5  
pytest-cov==3.0.0  
pytest-randomly==3.10.1  
pytz==2021.3  
PyYAML==5.4.1  
-e git+https://github.com/lemon24/reader.git@5542151591d17eaa1a06da405066f679c0844553#egg=reader  
readme-renderer==30.0  
requests==2.26.0  
requests-mock==1.9.3  
requests-toolbelt==0.9.1  
requests-wsgi-adapter==0.4.1  
rfc3986==1.5.0  
sgmllib3k==1.0.0  
six==1.16.0  
snowballstemmer==2.1.0  
sortedcontainers==2.4.0  
soupsieve==2.2.1  
Sphinx==4.2.0  
sphinx-click==3.0.1  
sphinx-hoverxref==0.7b1  
sphinx-issues==1.2.0  
sphinx-rtd-theme==1.0.0  
sphinxcontrib-applehelp==1.0.2  
sphinxcontrib-devhelp==1.0.2  
sphinxcontrib-htmlhelp==2.0.0  
sphinxcontrib-jsmath==1.0.1  
sphinxcontrib-log-cabinet==1.0.1  
sphinxcontrib-qthelp==1.0.3  
sphinxcontrib-serializinghtml==1.1.5  
toml==0.10.2  
tomli==1.2.1  
tox==3.24.4  
tqdm==4.62.3  
twine==3.4.2  
types-requests==2.25.9  
typing-extensions==3.10.0.2  
urllib3==1.26.7  
virtualenv==20.8.1  
webencodings==0.5.1  
Werkzeug==2.0.2  
zipp==3.6.0

What code are you running? Give us a specific commit of a specific repo that we can check out.
https://github.com/lemon24/reader/tree/5542151591d17eaa1a06da405066f679c0844553
Relevant Coverage config: https://github.com/lemon24/reader/blob/5542151591d17eaa1a06da405066f679c0844553/setup.cfg#L145-L154
To duplicate the environment, run this (in a virtualenv):
git clone https://github.com/lemon24/reader.git
cd reader
git reset --hard 5542151
pip install -e '.[search,cli,app,tests,docs,dev,unstable-plugins]'
What commands did you run?

$ coverage --version  
Coverage.py, version 6.0.1 with C extension  
Full documentation is at https://coverage.readthedocs.io  
$ rm -r .coverage  
$ pytest -k entry_dedupe --cov=reader.plugins.entry_dedupe  
============================= test session starts ==============================  
platform darwin -- Python 3.8.1, pytest-6.2.5, py-1.10.0, pluggy-1.0.0  
rootdir: /Users/lemon/code/reader, configfile: setup.cfg  
plugins: flaky-3.7.0, requests-mock-1.9.3, hypothesis-6.23.1, cov-3.0.0  
collected 1490 items / 1465 deselected / 25 selected  
tests/test_plugins_entry_dedupe.py .........................             [100%]  
---------- coverage: platform darwin, python 3.8.1-final-0 -----------  
Name                                 Stmts   Miss Branch BrPart  Cover  
----------------------------------------------------------------------  
src/reader/plugins/entry_dedupe.py     134     33     73      0    84%  
----------------------------------------------------------------------  
TOTAL                                  134     33     73      0    84%  
===================== 25 passed, 1465 deselected in 1.29s ======================

$ pytest -k entry_dedupe --cov=reader -q | grep entry_dedupe  
src/reader/plugins/entry_dedupe.py            134      0     73      0   100%

Expected behavior

100% coverage for entry_dedupe.py, as Coverage 5.5 reported previously:

$ pip install coverage==5.5
$ coverage --version                                       
Coverage.py, version 5.5 with C extension
Full documentation is at https://coverage.readthedocs.io
$ rm -r .coverage                                          
$ pytest -k entry_dedupe --cov=reader.plugins.entry_dedupe 
============================= test session starts ==============================
platform darwin -- Python 3.8.1, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /Users/lemon/code/reader, configfile: setup.cfg
plugins: flaky-3.7.0, requests-mock-1.9.3, hypothesis-6.23.1, cov-3.0.0
collected 1490 items / 1465 deselected / 25 selected                           

tests/test_plugins_entry_dedupe.py .........................             [100%]

---------- coverage: platform darwin, python 3.8.1-final-0 -----------
Name                                 Stmts   Miss Branch BrPart  Cover
----------------------------------------------------------------------
src/reader/plugins/entry_dedupe.py     134      0     73      0   100%
----------------------------------------------------------------------
TOTAL                                  134      0     73      0   100%


===================== 25 passed, 1465 deselected in 1.39s ======================