bpo-33735: Fix test_multiprocessing random failure by vstinner · Pull Request #8059 · python/cpython (original) (raw)

I validated on my laptop that the change fixes the false alarm.

I modified locally regrtest to make it stricter:

diff --git a/Lib/test/libregrtest/refleak.py b/Lib/test/libregrtest/refleak.py
index 6724488fcf..a3c50e21e0 100644
--- a/Lib/test/libregrtest/refleak.py
+++ b/Lib/test/libregrtest/refleak.py
@@ -101,7 +101,7 @@ def dash_R(the_module, test, indirect_test, huntrleaks):
     failed = False
     for deltas, item_name, checker in [
         (rc_deltas, 'references', check_rc_deltas),
-        (alloc_deltas, 'memory blocks', check_rc_deltas),
+        (alloc_deltas, 'memory blocks', check_fd_deltas),
         (fd_deltas, 'file descriptors', check_fd_deltas)
     ]:
         # ignore warmup runs

On master, I can easily reproduce the issue on my idle laptop:

$ ./python -m test test_multiprocessing_spawn -m test.test_multiprocessing_spawn.WithProcessesTestPool.test_imap_unordered -R 3:20
Run tests sequentially
0:00:00 load avg: 0.78 [1/1] test_multiprocessing_spawn
beginning 23 repetitions
12345678901234567890123
.......................
test_multiprocessing_spawn leaked [5, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] memory blocks, sum=8
test_multiprocessing_spawn failed

== Tests result: FAILURE ==

1 test failed:
    test_multiprocessing_spawn

Total duration: 11 sec 112 ms
Tests result: FAILURE

With the change, the bug goes away:

vstinner@apu$ ./python -m test test_multiprocessing_spawn -m test.test_multiprocessing_spawn.WithProcessesTestPool.test_imap_unordered -R 3:20
Run tests sequentially
0:00:00 load avg: 0.92 [1/1] test_multiprocessing_spawn
beginning 23 repetitions
12345678901234567890123
.......................
test_multiprocessing_spawn leaked [0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] memory blocks, sum=1
test_multiprocessing_spawn failed

== Tests result: FAILURE ==

1 test failed:
    test_multiprocessing_spawn

Total duration: 9 sec 967 ms
Tests result: FAILURE

The Refleak buildbots use -R 3:3, so only the first 3 numbers matter. Without the change, the first 3 numbers are non-zero, whereas they only one is non-zero with the change.