5.0.0 breaking changes: --append behavior · Issue #880 · nedbat/coveragepy (original) (raw)

Describe the bug

> Some breaking changes (?) should be documented in https://coverage.readthedocs.io/en/coverage-5.0/whatsnew5x.html

~~I'm assuming the --append option is now enabled by default, thus breaking codecov combine. (That's my guess, see below to make up your mind).~~

Edited: the inconsistency with 4.5.4 seem to be a bug fix.

I haven't seen this documented anywhere. Our test suite broke this morning, and after some troubleshooting It appears that's related to the 5.0.0 update of coverage.py.

We use coverage.py as part of our automated unit testing, here's our basic workflow:

Tests are started using tox
We run several unit tests using coverage run -a -m [...]
coverage combine
We call codecov which starts off by calling coverage xml

To Reproduce

What version of Python are you using?

gpotter@vmg:~/github/scapy$ python --version
Python 3.7.4

What version of coverage.py are you using? The output of coverage debug sys is helpful. For coverage 4.5.4

gpotter@vmg:~/github/scapy$ coverage debug sys -- sys ------------------------------------------------------- version: 4.5.4 coverage: /home/gpotter/.pyenv/versions/3.7.4/lib/python3.7/site-packages/coverage/init.py cover_paths: /netdisk/home/gpotter/.pyenv/versions/3.7.4/lib/python3.7/site-packages/coverage pylib_paths: /netdisk/home/gpotter/.pyenv/versions/3.7.4/lib/python3.7 tracer: CTracer plugins.file_tracers: -none- plugins.configurers: -none- config_files: .coveragerc configs_read: .coveragerc data_path: /netdisk/home/gpotter/github/scapy/.coverage python: 3.7.4 (default, Sep 11 2019, 21:46:22) [GCC 7.4.0] platform: Linux-5.0.0-1025-azure-x86_64-with-debian-buster-sid implementation: CPython executable: /home/gpotter/.pyenv/versions/3.7.4/bin/python3 cwd: /netdisk/home/gpotter/github/scapy path: /home/gpotter/.pyenv/versions/3.7.4/lib/python37.zip /home/gpotter/.pyenv/versions/3.7.4/lib/python3.7 /home/gpotter/.pyenv/versions/3.7.4/lib/python3.7/lib-dynload /home/gpotter/.local/lib/python3.7/site-packages /home/gpotter/.local/lib/python3.7/site-packages/mcstatus-2.2.1-py3.7.egg /home/gpotter/.local/lib/python3.7/site-packages/dnspython3-1.15.0-py3.7.egg /home/gpotter/.local/lib/python3.7/site-packages/Click-7.0-py3.7.egg /home/gpotter/.local/lib/python3.7/site-packages/six-1.12.0-py3.7.egg /home/gpotter/.local/lib/python3.7/site-packages/dnspython-1.15.0-py3.7.egg /home/gpotter/.pyenv/versions/3.7.4/lib/python3.7/site-packages environment: PYENV_DIR = /home/gpotter/github/scapy PYENV_HOOK_PATH = /home/gpotter/.pyenv/pyenv.d:/usr/local/etc/pyenv.d:/etc/pyenv.d:/usr/lib/pyenv/hooks:/home/gpotter/.pyenv/plugins/pyenv-virtualenv/etc/pyenv.d:/home/gpotter/.pyenv/plugins/pyenv-which-ext/etc/pyenv.d PYENV_ROOT = /home/gpotter/.pyenv PYENV_SHELL = bash PYENV_VERSION = 3.7.4 PYENV_VIRTUALENV_INIT = 1 command_line: /home/gpotter/.pyenv/versions/3.7.4/bin/coverage debug sys source_match: -none- source_pkgs_match: -none- include_match: -none- omit_match: /netdisk/home/gpotter/github/scapy/scapy/tools/UTscapy.py /netdisk/home/gpotter/github/scapy/test/* /netdisk/home/gpotter/github/scapy/scapy/modules/six.py /netdisk/home/gpotter/github/scapy/scapy/modules/winpcapy.py /netdisk/home/gpotter/github/scapy/scapy/modules/ethertypes.py /netdisk/home/gpotter/github/scapy/.tox/* /private/* cover_match: /netdisk/home/gpotter/.pyenv/versions/3.7.4/lib/python3.7/site-packages/coverage pylib_match: /netdisk/home/gpotter/.pyenv/versions/3.7.4/lib/python3.7

For coverage 5.0.0

gpotter@vmg:~/github/scapy$ coverage debug sys -- sys ------------------------------------------------------- version: 5.0 coverage: /home/gpotter/.pyenv/versions/3.7.4/lib/python3.7/site-packages/coverage/init.py tracer: -none- CTracer: available plugins.file_tracers: -none- plugins.configurers: -none- plugins.context_switchers: -none- configs_attempted: .coveragerc configs_read: .coveragerc config_file: .coveragerc config_contents: '[run]\nconcurrency = multiprocessing\nomit =\n # Scapy specific paths\n scapy/tools/UTscapy.py\n test/*\n # Scapy external modules\n scapy/modules/six.py\n scapy/modules/winpcapy.py\n scapy/modules/ethertypes.py\n # .tox specific path\n .tox/*\n # OS specific paths\n /private/*\n' data_file: -none- python: 3.7.4 (default, Sep 11 2019, 21:46:22) [GCC 7.4.0] platform: Linux-5.0.0-1025-azure-x86_64-with-debian-buster-sid implementation: CPython executable: /home/gpotter/.pyenv/versions/3.7.4/bin/python3 def_encoding: utf-8 fs_encoding: utf-8 pid: 91527 cwd: /netdisk/home/gpotter/github/scapy path: /netdisk/home/gpotter/.pyenv/versions/3.7.4/bin /home/gpotter/.pyenv/versions/3.7.4/lib/python37.zip /home/gpotter/.pyenv/versions/3.7.4/lib/python3.7 /home/gpotter/.pyenv/versions/3.7.4/lib/python3.7/lib-dynload /home/gpotter/.local/lib/python3.7/site-packages /home/gpotter/.local/lib/python3.7/site-packages/mcstatus-2.2.1-py3.7.egg /home/gpotter/.local/lib/python3.7/site-packages/dnspython3-1.15.0-py3.7.egg /home/gpotter/.local/lib/python3.7/site-packages/Click-7.0-py3.7.egg /home/gpotter/.local/lib/python3.7/site-packages/six-1.12.0-py3.7.egg /home/gpotter/.local/lib/python3.7/site-packages/dnspython-1.15.0-py3.7.egg /home/gpotter/.pyenv/versions/3.7.4/lib/python3.7/site-packages environment: PYENV_DIR = /home/gpotter/github/scapy PYENV_HOOK_PATH = /home/gpotter/.pyenv/pyenv.d:/usr/local/etc/pyenv.d:/etc/pyenv.d:/usr/lib/pyenv/hooks:/home/gpotter/.pyenv/plugins/pyenv-virtualenv/etc/pyenv.d:/home/gpotter/.pyenv/plugins/pyenv-which-ext/etc/pyenv.d PYENV_ROOT = /home/gpotter/.pyenv PYENV_SHELL = bash PYENV_VERSION = 3.7.4 PYENV_VIRTUALENV_INIT = 1 command_line: /home/gpotter/.pyenv/versions/3.7.4/bin/coverage debug sys sqlite3_version: 2.6.0 sqlite3_sqlite_version: 3.22.0

~~What versions of what packages do you have installed? The output of pip freeze is helpful.~~
What code are you running? (Most likely not useful,) this commit
What commands did you run?

coverage --version

those commands are a demo: we just start scapy and stop it. You can most likely use anything

echo "exit()" | coverage run -a -m scapy > /dev/null 2>&1 echo "exit()" | coverage run -a -m scapy > /dev/null 2>&1 ls -alF | grep ".coverage" coverage combine echo $?

Expected behavior

Here's what happens when running the code displayed above:

On 5.0.0:

gpotter@vmg:~~/github/scapy$ coverage --version Coverage.py, version 5.0 with C extension Full documentation is at https://coverage.readthedocs.io gpotter@vmg:~~/github/scapy$ echo "exit()" | coverage run -a -m scapy > /dev/null 2>&1 gpotter@vmg:~~/github/scapy$ echo "exit()" | coverage run -a -m scapy > /dev/null 2>&1 gpotter@vmg:~~/github/scapy$ ll | grep ".coverage" -rw-r--r-- 1 gpotter gpotter 81920 Dec 15 13:29 .coverage -rw-rw-r-- 1 gpotter gpotter 299 Dec 6 22:41 .coveragerc gpotter@vmg:~~/github/scapy$ coverage combine No data to combine gpotter@vmg:~~/github/scapy$ echo $? 1

On 4.5.4:

gpotter@vmg:~~/github/scapy$ coverage --version Coverage.py, version 4.5.4 with C extension Documentation at https://coverage.readthedocs.io gpotter@vmg:~~/github/scapy$ echo "exit()" | coverage run -a -m scapy > /dev/null 2>&1 gpotter@vmg:~~/github/scapy$ echo "exit()" | coverage run -a -m scapy > /dev/null 2>&1 gpotter@vmg:~~/github/scapy$ ll | grep ".coverage" -rw-rw-r-- 1 gpotter gpotter 82949 Dec 15 13:30 .coverage.vmg.90662.854455 -rw-rw-r-- 1 gpotter gpotter 82949 Dec 15 13:30 .coverage.vmg.90762.195813 -rw-rw-r-- 1 gpotter gpotter 299 Dec 6 22:41 .coveragerc gpotter@vmg:~~/github/scapy$ coverage combine gpotter@vmg:~~/github/scapy$ echo $? 0

Our `.coveragerc` file

[run]
concurrency = multiprocessing
omit =
    # Scapy specific paths
    scapy/tools/UTscapy.py
    test/*
    # Scapy external modules
    scapy/modules/six.py
    scapy/libs/winpcapy.py
    scapy/libs/ethertypes.py
    # .tox specific path
    .tox/*
    # OS specific paths
    /private/*

As shown, the return code is 1 on 5.0.0, and 0 on 4.5.4. This explains why our tests suddently failed starting today.

Should this be documented ? Is this behavior expected ?

Thanks for your time