Issue 13131: FD leak in urllib2 (original) (raw)

Created on 2011-10-08 12:25 by Valery.Khamenya, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (7)
msg145166 - (view)	Author: Valery Khamenya (Valery.Khamenya)	Date: 2011-10-08 12:25
Explanation from dablitz's comment at https://bugs.pypy.org/issue867 : urllib2 in the stdlib leaks fd's if an exception is raised while opening a connection. The issue occurs due to a socket being opened then an exception being raised before an object with the socket is returned, leaving no way to explicitly close the object. On cpython this would not be an issue as the object would lose all references almost immediately however it lingers around with a proper GC causing FD's to build up if the same condition happens repeatedly (eg a loop/web crawling) The file enclosed is a script to generate the leakage, to run invok it as follows leak.py pypy should start leaking FD's and can be see in /proc//fd Related issues: http://bugs.python.org/issue3066 http://bugs.python.org/issue1208304 http://bugs.python.org/issue1601399
msg145169 - (view)	Author: Valery Khamenya (Valery.Khamenya)	Date: 2011-10-08 13:23
by the way, timeout parameter should be set to 0.2 as for my 13Mbit ADSL line. With 0.002 it is not reproducible for my environment
msg145183 - (view)	Author: Valery Khamenya (Valery.Khamenya)	Date: 2011-10-08 19:25
the problem seems to be fixed with the patch attached. Thanks go to fijal@freenode
msg145184 - (view)	Author: Ezio Melotti (ezio.melotti) *	Date: 2011-10-08 19:50
Pay attention not to introduce regressions like the one in #12576 while fixing this. I'm not sure there are similar tests for urllib2 -- if not they should be added.
msg145568 - (view)	Author: Terry J. Reedy (terry.reedy) *	Date: 2011-10-14 22:34
Can you test both the leak and fix (if needed) on 3.2? (I am on Win7).
msg145668 - (view)	Author: Éric Araujo (eric.araujo) *	Date: 2011-10-17 11:40
Instead of except+raise, you could use finally.
msg355318 - (view)	Author: STINNER Victor (vstinner) *	Date: 2019-10-24 10:29
This issue is a duplicate of bpo-12133 which has been fixed in Python 2.7 by: commit c74a6ba2d6c1f331896cf8dacc698b0b88c7e670 Author: Victor Stinner <victor.stinner@haypocalc.com> Date: Fri Jun 17 14:06:27 2011 +0200 Issue #12133: AbstractHTTPHandler.do_open() of urllib.request closes the HTTP connection if its getresponse() method fails with a socket error. Patch written by Ezio Melotti.