| msg322444 - (view) |
Author: Tim Golden (tim.golden) *  |
Date: 2018-07-26 19:22 |
| test_mmap currently uses the test.support.TESTFN functionality which creates a temporary file local to the test directory named around the pid. This can give rise to race conditions where tests are competing with each other to delete and recreate the file. |
|
|
| msg322456 - (view) |
Author: Eryk Sun (eryksun) *  |
Date: 2018-07-27 02:25 |
| > tests are competing with each other to delete and recreate the file When is this an issue? Each parallel test process should have its own working directory under "{TEMPDIR}/test_python_{pid}". |
|
|
| msg322464 - (view) |
Author: Eryk Sun (eryksun) * |
Date: 2018-07-27 05:39 |
| To clarify, TEMPDIR in this case is from the following code in Lib/test/libregrtest/main.py: # When tests are run from the Python build directory, it is best practice # to keep the test files in a subfolder. This eases the cleanup of leftover # files using the "make distclean" command. if sysconfig.is_python_build(): TEMPDIR = sysconfig.get_config_var('abs_builddir') if TEMPDIR is None: # bpo-30284: On Windows, only srcdir is available. Using abs_builddir # mostly matters on UNIX when building Python out of the source tree, # especially when the source tree is read only. TEMPDIR = sysconfig.get_config_var('srcdir') TEMPDIR = os.path.join(TEMPDIR, 'build') else: TEMPDIR = tempfile.gettempdir() TEMPDIR = os.path.abspath(TEMPDIR) Then in class Regrtest we have: def main(self, tests=None, **kwargs): global TEMPDIR if sysconfig.is_python_build(): try: os.mkdir(TEMPDIR) except FileExistsError: pass # Define a writable temp dir that will be used as cwd while running # the tests. The name of the dir includes the pid to allow parallel # testing (see the -j option). test_cwd = 'test_python_{}'.format(os.getpid()) test_cwd = os.path.join(TEMPDIR, test_cwd) # Run the tests in a context manager that temporarily changes the CWD to a # temporary and writable directory. If it's not possible to create or # change the CWD, the original CWD will be used. The original CWD is # available from support.SAVEDCWD. with support.temp_cwd(test_cwd, quiet=True): self._main(tests, kwargs) |
|
|
| msg322471 - (view) |
Author: Tim Golden (tim.golden) * |
Date: 2018-07-27 07:56 |
| Thanks for the information, eryksun. For the moment, I can only say with a fair degree of certainty that using the tempfile functions as I have in test_bz2 & test_mmap appears to solve the issue which is repeatably if intermittently present without that change. I'll be trying to narrow the issue down before making any further changes |
|
|
| msg370288 - (view) |
Author: Cheryl Sabella (cheryl.sabella) * |
Date: 2020-05-29 10:15 |
| @tim.golden, I closed this pull request as it was from an unknown repository and new pull request would need to be created with the changes in order to move it forward. |
|
|
| msg370290 - (view) |
Author: TJG (tjguk) |
Date: 2020-05-29 10:18 |
| Thanks, Cheryl. It was getting pretty stale. If I think it's worthwhile I'll resurrect it. I suspect what happened was that my cpython fork got borked somehow between then and now. I probably ditched the repo and re-forked. That's for doing the housekeeping work here. |
|
|
| msg370295 - (view) |
Author: TJG (tjguk) |
Date: 2020-05-29 10:43 |
| That last should clearly have said: Thanks for doing the housekeeping work here :) |
|
|
| msg404762 - (view) |
Author: Tim Golden (tim.golden) * |
Date: 2021-10-22 12:40 |
| Closing as no longer reproducible in the current codebase on my current laptop |
|
|