| msg76158 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2008-11-21 02:28 |
| A refleak test of r67311 py3k shows several leaks: test_distutils leaked [142, 142, 142, 142] references, sum=568 test_httpservers leaked [0, 0, 0, 217] references, sum=217 test_multiprocessing leaked [0, 0, 24, 0] references, sum=24 test_pickle leaked [1, 1, 1, 1] references, sum=4 test_pickletools leaked [1, 1, 1, 1] references, sum=4 test_telnetlib leaked [-84, 84, 0, 0] references, sum=0 test_threadedtempfile leaked [94, -94, 0, 0] references, sum=0 |
|
|
| msg76159 - (view) |
Author: Christian Heimes (christian.heimes) * |
Date: 2008-11-21 02:40 |
| Only distutils and pickle seem to have real leaks. $ ./python Lib/test/regrtest.py -R:15: test_multiprocessing test_multiprocessing beginning 20 repetitions 12345678901234567890 .................... 1 test OK. [124096 refs] $ ./python Lib/test/regrtest.py -R:15: test_distutils test_distutils beginning 20 repetitions 12345678901234567890 .................... test_distutils leaked [144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144] references, sum=2160 1 test OK. [123507 refs] $ ./python Lib/test/regrtest.py -R:15: test_httpservers [...] test_httpservers leaked [-217, 0, 198, 19, -35, 20, -202, 217, -217, 217, 0, 0, 0, 0, -33] references, sum=-33 1 test OK. [96060 refs] $ ./python Lib/test/regrtest.py -R:15: test_pickle test_pickle beginning 20 repetitions 12345678901234567890 .................... test_pickle leaked [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] references, sum=15 1 test OK. [96783 refs] |
|
|
| msg76161 - (view) |
Author: Christian Heimes (christian.heimes) * |
Date: 2008-11-21 02:50 |
| distutils.tests.test_build_ext leaks the references. I think it's related to the xx module and totally harmless. |
|
|
| msg76162 - (view) |
Author: Christian Heimes (christian.heimes) * |
Date: 2008-11-21 02:59 |
| This simple patch doesn't load the 'xx' module more than once in ref leak tests. The problem may also be caused by the xxmodule itself. Somebody should give it a proper review :) |
|
|
| msg76167 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2008-11-21 09:07 |
| I have studied this some time ago. The xx module comes each time from a different file (in a directory created with tempfile.mkdtemp). Every instance of the module creates a new entry in the static "extensions" dictionary in Python/import.c. This entry at least contains a copy of the module dictionary, which explains the number of "leaked" references. I do not see any way to clear this dictionary. The proposed patch is probably the best thing to do. |
|
|
| msg76169 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2008-11-21 09:48 |
| The refleak in test_pickle comes from unpickler_read(). The attached patch corrects it. |
|
|
| msg76172 - (view) |
Author: Christian Heimes (christian.heimes) * |
Date: 2008-11-21 10:24 |
| Good analysis Amaury! The new patch for build ext uses a single temp dir. |
|
|
| msg76175 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2008-11-21 10:57 |
| The problem is that on Windows (and cygwin) you cannot unlink a .pyd that is currently loaded in memory. I tried to use ctypes and call FreeLibrary. Now the .pyd can be removed, but the interpreter crashes when it comes to free the module on shutdown. I'm afraid that until python has a real support for dlclose() on dynamic loaded module, the best is to skip this test during the refleak hunt. |
|
|
| msg76422 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2008-11-25 19:33 |
| Amaury's patch for pickle looks fine to me. |
|
|
| msg76428 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2008-11-25 21:12 |
| Applied pickle-leak2.patch in 67381. |
|
|
| msg76431 - (view) |
Author: Christian Heimes (christian.heimes) * |
Date: 2008-11-25 21:21 |
| Second fix applied in r67382 |
|
|