| msg301058 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2017-08-31 20:18 |
| Copy attached test_leak.py to Lib/test/ and run: haypo@selma$ ./python -m test -R 3:30 test_leak Run tests in parallel using 1 child processes 0:00:00 load avg: 0.67 [1/1/1] test_leak failed beginning 33 repetitions 123456789012345678901234567890123 ................................. test_leak leaked [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] memory blocks, sum=30 1 test failed: test_leak Total duration: 682 ms Tests result: FAILURE It seems like Thread2 constructor creates a dictionary on "self.hashvalue = (...)". It's a dictionary sharing keys, but it seems like the dict table memory is not released whne the bug occurs. Sadly, I'm only able to reproduce the bug on a 32-bit build. Since the bug depends on exact hash values, maybe the bug exists in 64-bit builds, but is less likely? I'm not 100% sure if the bug is related to dict or set. The bug comes from bpo-31217. |
|
|
| msg301060 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-08-31 20:22 |
| If you cannot reproduce the bug on 32-bit build, try to set "FIXED_HASH = 0" in test_leak.py, run "./python -m test -R 3:10 test_leak" multiple times until it fails, then copy the last value of the printed list into hash_values in test_leak.py. |
|
|
| msg301062 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-08-31 21:25 |
| I spent a lot of time in gdb, but I'm not sure anymore that the issue is related to dict shared keys. |
|
|
| msg301063 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2017-08-31 22:03 |
| It's a really weird bug you found. It depends on the exact values passed to -R: $ ./python -m test -R 3:6 test_haypoleak Run tests sequentially 0:00:00 load avg: 0.39 [1/1] test_haypoleak beginning 9 repetitions 123456789 ......... test_haypoleak leaked [1, 1, 1, 1, 1, 1] memory blocks, sum=6 test_haypoleak failed $ ./python -m test -R 2:6 test_haypoleak Run tests sequentially 0:00:00 load avg: 0.36 [1/1] test_haypoleak beginning 8 repetitions 12345678 ........ 1 test OK. |
|
|
| msg301064 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2017-08-31 22:04 |
| Actually, it's not the bug that is weird, but the heuristics in -R seem completely weird now. If I print the alloc and rc deltas: $ ./python -m test -R 3:6 test_haypoleak Run tests sequentially 0:00:00 load avg: 0.09 [1/1] test_haypoleak beginning 9 repetitions 123456789 ......... alloc: [0, 0, 0, 1, 1, 1, 1, 1, 1], rc: [0, 0, 0, 0, 0, 0, 0, 0, 0] test_haypoleak leaked [1, 1, 1, 1, 1, 1] memory blocks, sum=6 test_haypoleak failed $ ./python -m test -R 2:6 test_haypoleak Run tests sequentially 0:00:00 load avg: 0.10 [1/1] test_haypoleak beginning 8 repetitions 12345678 ........ alloc: [0, 0, 0, 1, 1, 1, 1, 1], rc: [0, 0, 0, 0, 0, 0, 0, 0] 1 test OK. |
|
|
| msg301065 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2017-08-31 22:05 |
| s/completely weird/broken/ $ ./python -m test -R 2:20 test_haypoleak Run tests sequentially 0:00:00 load avg: 0.10 [1/1] test_haypoleak beginning 22 repetitions 1234567890123456789012 ...................... alloc: [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], rc: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] 1 test OK. |
|
|
| msg301067 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2017-08-31 22:11 |
| To come back to this bug, it still fails if add a __slots__, so I don't think it's related to shared key dictionaries. |
|
|
| msg301068 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-08-31 22:23 |
| Antoine: "Actually, it's not the bug that is weird, but the heuristics in -R seem completely weird now. If I print the alloc and rc deltas:" I changed it recently to reduce false alarms: see bpo-30776. Antoine: "To come back to this bug, it still fails if add a __slots__, so I don't think it's related to shared key dictionaries." Oh ok, in that case, I will move back to bpo-31217. I close this issue as a duplicate. |
|
|