| msg32364 - (view) |
Author: Hirokazu Yamamoto (ocean-city) *  |
Date: 2007-06-18 14:43 |
| Hello. I investigated ref leak report related to thread. Please run python regrtest.py -R :: test_leak.py (attached file) Sometimes ref leak is reported. # I saw this as regression failure on python-checkins. # total ref count 92578 -> 92669 _Condition 2 Thread 6 _Event 1 bool 10 instancemethod 1 code 2 dict 9 file 1 frame 3 function 2 int 1 list 2 builtin_function_or_method 5 NoneType 2 str 27 thread.lock 7 tuple 5 type 5 Probably this happens because threading.Thread is implemented as Python code, (expecially threading.Thread#join), the code of regrtest.py if i >= nwarmup: deltas.append(sys.gettotalrefcount() - rc - 2) can run before thread really quits. (before Moudles/threadmodule.c t_bootstrap()'s Py_DECREF(boot->func); Py_DECREF(boot->args); Py_XDECREF(boot->keyw); runs) So I experimentally inserted the code to wait for thread termination. (attached file experimental.patch) And I confirmed error was gone. # Sorry for hackish patch which only runs on windows. It should run # on other platforms if you replace Sleep() in Python/sysmodule.c # sys_debug_ref_leak_leave() with appropriate function. |
|
|
| msg32365 - (view) |
Author: Hirokazu Yamamoto (ocean-city) * |
Date: 2007-06-18 17:48 |
| Sorry, I updated the patch. Moved fprintf(stderr, "================= thread leave\n"); current_thread_count--; in threadmodule.c just before PyThread_exit_thread(). File Added: archive.zip |
|
|
| msg32366 - (view) |
Author: Hirokazu Yamamoto (ocean-city) * |
Date: 2007-06-19 01:13 |
| Umm, maybe still not enough? applied this patch (fix for 3 str leaks) test_urllib2_localnet.py still reports leaks sometimes. Index: Lib/test/test_urllib2_localnet.py =================================================================== --- Lib/test/test_urllib2_localnet.py (revision 56002) +++ Lib/test/test_urllib2_localnet.py (working copy) @@ -208,7 +208,7 @@ testing. """ - digest_auth_handler = DigestAuthHandler() + digest_auth_handler = None def log_message(self, format, *args): # Uncomment the next line for debugging. @@ -240,6 +240,7 @@ PROXY_URL = "http://127.0.0.1:%d" % PORT def setUp(self): + FakeProxyHandler.digest_auth_handler = DigestAuthHandler() FakeProxyHandler.digest_auth_handler.set_users({ self.USER : self.PASSWD }) @@ -257,6 +258,7 @@ def tearDown(self): self.server.stop() + FakeProxyHandler.digest_auth_handler = None def test_proxy_with_bad_password_raises_httperror(self): self._digest_auth_handler.add_password(self.REALM, self.URL, /////////////////////////// test_urllib2_localnet leaked [2, 0, 0, 0] references, sum=2 # it is now rare case though.... |
|
|
| msg85622 - (view) |
Author: Daniel Diniz (ajaksu2) *  |
Date: 2009-04-06 09:39 |
| Attaching Hirokazu's tests and experimental patch as plaintext. |
|
|
| msg87547 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2009-05-10 21:01 |
| Hmm, rather than some C code in the internals, couldn't we simply iterate over the registered Thread objects and join() on those which are still running? |
|
|
| msg94741 - (view) |
Author: Hirokazu Yamamoto (ocean-city) * |
Date: 2009-10-31 03:32 |
| This issue seems to be fixed in r75958. |
|
|