Issue 22390: test.regrtest should complain if a test doesn't remove temporary files (original) (raw)

Created on 2014-09-11 14:30 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
regrtest_warn_lost_files.patch serhiy.storchaka,2014-09-11 16:11 review
fix_tests.patch vstinner,2014-09-17 22:13 review
regrtest_warn_lost_files2.patch serhiy.storchaka,2014-10-13 08:23 review
Messages (23)
msg226777 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-11 14:30
A change in test_glob of issue #13968 started to fail because a previous test created temporary files but didn't remove them. test.regrtest should at least emit a warning if the temporary directory used to run tests is not empty before removing it.
msg226786 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-11 16:11
Here is a patch. It warns if new files or directories are left after test run in current directory and removes those of them which starts with TESTFN (i.e. TESTFN_UNICODE, TESTFN + "2", TESTFN + ".py" and other names used in tests).
msg226787 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-11 16:19
I would prefer to fix tests instead of trying to remove arbitrary files. I'm not sure that "fn.startswith(support.TESTFN)" check is safe enough. Maybe we should not remove TESTFN neither. You are supposed to be able to run tests without regrtest.
msg226788 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-11 16:20
I ran the test suite with the patch applied: 7 tests altered the execution environment: test_imp test_import test_pdb test_posix test_source_encoding test_support test_threaded_import I will try to investigate these warnings.
msg226792 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-11 17:23
> I would prefer to fix tests instead of trying to remove arbitrary files. I'm > not sure that "fn.startswith(support.TESTFN)" check is safe enough. This allow other tests which leaks the same file to be reported too. support.TESTFN contains process ID so it is unique enough.
msg226793 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-11 17:37
> I will try to investigate these warnings. Run tests with the -vv option.
msg227015 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-17 22:13
fix_tests.patch: Fix the 7 tests which create files without removing them. 4 tests (test_imp, test_pdb, test_source_encoding and test_support) create a __pycache__ directory. Maybe this directory should be ignored by regrtest?
msg228584 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-10-05 15:39
New changeset 8cf8bff3569e by Victor Stinner in branch '3.4': Issue #22390: Remove files created by tests https://hg.python.org/cpython/rev/8cf8bff3569e
msg229196 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-10-12 19:34
How about regrtest_warn_lost_files.patch?
msg229200 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-10-12 22:03
> This allow other tests which leaks the same file to be reported too. I don't understand your answer. The "fn.startswith(support.TESTFN)" test is not used in get_files(). If I understood correctly, your patch changes two things: - it now reports files created for tests but not deleted (purpose of this issue) - it removes more files than before I don't understand why you want to remove more files than before. You may open a different issue, or at least explain the rationale. I never see any forgotten test file after running tests, so I don't see why you are worried because of them. And with your first patch, we will now noticed forgotten files, so we can just fix tests.
msg229234 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-10-13 08:23
> I don't understand why you want to remove more files than before. You may > open a different issue, or at least explain the rationale. I thought it would be good idea slightly extend this cleanup while we are here. I'm not motivated enough to open a different issue. Well, here is a patch which removes only TESTFN. It is still improved, uses support.unlink and support.rmtree instead of os.unlink and shutil.rmtree. You can just drop cleanup code at all if you prefer. All is good to me. > I never see any forgotten test file after running tests, so I don't see why > you are worried because of them. This is because regrtest creates temporary directory and goes to it. But when you execute Python test directly, test files are created in the current directory. > And with your first patch, we will now > noticed forgotten files, so we can just fix tests. But we will noticed only one about test at the time if several tests forgot the same file. This will needed several iterations.
msg236080 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-16 00:19
What about this issue?
msg239488 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-29 12:42
Victor?
msg239525 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-29 21:48
regrtest_warn_lost_files2.patch looks good to me. I just ran the test suite with "./python -m test -j0 -rW" on Linux, I didn't get any warning. When I modified test_os to create a file "x", it was noticed by your patch. Go ahead.
msg239527 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-29 22:31
New changeset f40984e7ceea by Serhiy Storchaka in branch '2.7': Issue #22390: test.regrtest now emits a warning if temporary files or https://hg.python.org/cpython/rev/f40984e7ceea New changeset 05e6bab4db8f by Serhiy Storchaka in branch '3.4': Issue #22390: test.regrtest now emits a warning if temporary files or https://hg.python.org/cpython/rev/05e6bab4db8f New changeset bed806c9eb4c by Serhiy Storchaka in branch 'default': Issue #22390: test.regrtest now emits a warning if temporary files or https://hg.python.org/cpython/rev/bed806c9eb4c
msg239529 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-29 23:07
New changeset 207db4338706 by Victor Stinner in branch '2.7': Issue #22390: Fix typo in regrtest, support => test_support https://hg.python.org/cpython/rev/207db4338706
msg239532 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-29 23:20
New changeset fb5c3528d0d7 by Victor Stinner in branch '2.7': Issue #22390: Fix test_aifc to remove the created file https://hg.python.org/cpython/rev/fb5c3528d0d7
msg239533 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-29 23:25
New changeset c5c31adbefeb by Victor Stinner in branch '2.7': Issue #22390: Fix test_pdb to remove created bar.pyc file https://hg.python.org/cpython/rev/c5c31adbefeb
msg239534 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-29 23:36
New changeset 33e48141d83f by Victor Stinner in branch '2.7': Issue #22390: Fix test_gzip, remove temporary file https://hg.python.org/cpython/rev/33e48141d83f
msg239536 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-30 00:04
On Windows, test_idle modifies os.environ: TCL_xxx and TIX_xxx (sorry for "xxx", I don't remember the full variable name) are added. And test_platform modifies os.environ['PATH'].
msg239539 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-30 00:21
New changeset 6ef2cacec2e9 by Victor Stinner in branch '2.7': Issue #22390: Fix test_gzip if unicode filename doesn't work https://hg.python.org/cpython/rev/6ef2cacec2e9
msg239575 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-30 07:10
Thank you for fixing backported patch and tests Victor.
msg239582 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-30 08:41
Serhiy Storchaka added the comment: > Thank you for fixing backported patch and tests Victor. No problem, thanks for your enhancement of regrtest ;-) I proposed the idea and you implemented it, great team work!
History
Date User Action Args
2022-04-11 14:58:07 admin set github: 66584
2015-03-30 08:41:08 vstinner set messages: +
2015-03-30 07:10:41 serhiy.storchaka set status: open -> closedmessages: + assignee: serhiy.storchakaresolution: fixedstage: patch review -> resolved
2015-03-30 00:21:08 python-dev set messages: +
2015-03-30 00:04:14 vstinner set messages: +
2015-03-29 23:36:27 python-dev set messages: +
2015-03-29 23:25:08 python-dev set messages: +
2015-03-29 23:20:07 python-dev set messages: +
2015-03-29 23:07:42 python-dev set messages: +
2015-03-29 22:31:22 python-dev set messages: +
2015-03-29 21:48:07 vstinner set messages: +
2015-03-29 12:42:22 serhiy.storchaka set messages: +
2015-02-16 00:19:00 serhiy.storchaka set messages: +
2014-10-13 08:23:05 serhiy.storchaka set files: + regrtest_warn_lost_files2.patchmessages: +
2014-10-12 22:03:46 vstinner set messages: +
2014-10-12 19:34:44 serhiy.storchaka set messages: +
2014-10-05 15:39:12 python-dev set nosy: + python-devmessages: +
2014-09-17 22:13:02 vstinner set files: + fix_tests.patchmessages: +
2014-09-11 17:37:09 serhiy.storchaka set messages: +
2014-09-11 17:23:24 serhiy.storchaka set messages: +
2014-09-11 16:20:12 vstinner set messages: +
2014-09-11 16:19:22 vstinner set messages: +
2014-09-11 16:11:09 serhiy.storchaka set files: + regrtest_warn_lost_files.patchtype: enhancementversions: + Python 2.7, Python 3.4keywords: + patchnosy: + pitrou, ezio.melotti, michael.foordmessages: + stage: patch review
2014-09-11 14:30:52 vstinner create