Relative paths duplicates paths on Windows · Issue #1752 · nedbat/coveragepy (original) (raw)
Describe the bug
I've been working on issues with nox's test suite. We weren't correctly including runs from all three OS's, and we have just been fixing that. We seem to have a 50% chance of failure with D:\a\nox\nox\__init__.py showing up with "no source for file". I'm pretty sure this started showing up for the first time without this fix roughly 1-2 days ago, but the issue is more reliable if we properly make sure coverage files don't overwrite each other. We are using editable installs and relative paths, so I'm not sure why it's getting stored that way (even in the succeeding 50%). If I look at the report on Windows, every file is listed twice. If I look at the .coverage.win32* files, I see that every file is present twice in files, once as a relative path, and once as an absolute path. The other OS's are just fine, it's only Windows that is doing this.
This is with a hatchling editable install of nox, inside a nox runner. Using pytest-cov.
To Reproduce
How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:
- What version of Python are you using? 3.7-3.12.
- What version of coverage.py shows the problem? The output of
coverage debug sysis helpful. 7.4.2 and 7.2.7 - What versions of what packages do you have installed? The output of
pip freezeis helpful. all latest supported - What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix. wntrblm/nox@3b8e026 passed at one point, and now is failing. wntrblm/nox@1c998d1 is probably better, though, since that's with the OS naming fix.
- What commands should we run to reproduce the problem? Be specific. Include everything, even
git clone,pip install, and so on. Explain like we're five! _I've only done it in CI, since I don't have easy access to a Windows machine right now, but it should be possible to runpipx runnox -s 'tests(python="3.12", tox_version="latest")'` (AFAICT it happens on all of them).
Expected behavior
There should not be two copies of files on Windows. SELECT * FROM file; reports:
1|D:\a\nox\nox\nox\__init__.py
2|D:\a\nox\nox\nox\_options.py
3|D:\a\nox\nox\nox\_option_set.py
4|D:\a\nox\nox\nox\tasks.py
5|D:\a\nox\nox\nox\registry.py
6|D:\a\nox\nox\nox\_decorators.py
7|D:\a\nox\nox\nox\_typing.py
8|D:\a\nox\nox\nox\_version.py
9|D:\a\nox\nox\nox\logger.py
10|D:\a\nox\nox\nox\manifest.py
11|D:\a\nox\nox\nox\sessions.py
12|D:\a\nox\nox\nox\command.py
13|D:\a\nox\nox\nox\popen.py
14|D:\a\nox\nox\nox\virtualenv.py
15|D:\a\nox\nox\nox\_parametrize.py
16|D:\a\nox\nox\nox\tox_to_nox.py
17|D:\a\nox\nox\nox\workflow.py
18|D:\a\nox\nox\nox\__main__.py
19|nox\__init__.py
20|nox\_options.py
21|nox\_option_set.py
22|nox\tasks.py
23|nox\registry.py
24|nox\_decorators.py
25|nox\_version.py
26|nox\logger.py
27|nox\manifest.py
28|nox\sessions.py
29|nox\command.py
30|nox\popen.py
31|nox\virtualenv.py
32|nox\_parametrize.py
33|nox\__main__.py
34|nox\workflow.py
35|nox\tox_to_nox.py
And the coverage report looks like this:
---------- coverage: platform win32, python 3.8.10-final-0 -----------
Name Stmts Miss Branch BrPart Cover
-------------------------------------------------------
nox\__init__.py 8 0 0 0 100%
nox\__init__.py 8 0 0 0 100%
nox\__main__.py 20 0 4 0 100%
nox\__main__.py 20 0 4 0 100%
nox\_decorators.py 52 0 6 0 100%
nox\_decorators.py 52 0 6 0 100%
nox\_option_set.py 108 0 38 0 100%
nox\_option_set.py 108 0 38 0 100%
nox\_options.py 78 0 32 0 100%
nox\_options.py 78 0 32 0 100%
nox\_parametrize.py 75 0 36 0 100%
nox\_parametrize.py 75 0 36 0 100%
nox\_typing.py 4 4 0 0 0%
nox\_version.py 42 1 14 1 96%
nox\_version.py 42 1 14 1 96%
nox\command.py 68 1 34 2 97%
nox\command.py 68 1 34 2 97%
nox\logger.py 47 0 16 1 98%
nox\logger.py 47 0 16 1 98%
nox\manifest.py 150 0 84 0 100%
nox\manifest.py 150 0 84 0 100%
nox\popen.py 38 7 8 1 83%
nox\popen.py 38 7 8 1 83%
nox\registry.py 23 0 6 0 100%
nox\registry.py 23 0 6 0 100%
nox\sessions.py 303 0 96 0 100%
nox\sessions.py 303 0 96 0 100%
nox\tasks.py 152 0 56 0 100%
nox\tasks.py 152 0 56 0 100%
nox\tox_to_nox.py 64 30 28 2 43%
nox\tox_to_nox.py 64 30 28 2 43%
nox\virtualenv.py 211 3 70 5 97%
nox\virtualenv.py 211 3 70 5 97%
nox\workflow.py 18 1 6 1 92%
nox\workflow.py 18 1 6 1 92%
-------------------------------------------------------
TOTAL 2918 90 1068 26 96%
Linux and macOS jobs do not show duplicate copies.
Additional context
I think I have finally been able to work around it by avoiding relative paths and using (I get an issue with macOS paths if I include it, though the duplicated paths issue is still fine). I'm pretty sure that any time I specify relative paths, this is going to be duplicated, and if I don't, it's fine. I've been trying a few dozen things, across several PRs if any of the other combinations are useful.source_pgks and avoiding the editable install
I think I can work around it by post-processing the data files to remove the D:\ files (I also swap \ for / - if I do that locally, I can view the files without the duplicate lines).
This is my working workaround in wntrblm/nox#778:
if sys.platform.startswith("win"): with contextlib.closing(sqlite3.connect(coverage_file)) as con, con: con.execute("UPDATE file SET path = REPLACE(path, '\', '/')") con.execute("DELETE FROM file WHERE SUBSTR(path, 2, 1) == ':'")