[Python-Dev] Track ResourceWarning warnings with tracemalloc (original) (raw)

Victor Stinner victor.stinner at gmail.com
Fri Nov 29 13:12:26 CET 2013

Hi,

I'm trying to write an example of usage of the new tracemalloc.get_object_traceback() function. Last month I proposed to use it to give the traceback where a file/socket was allocated on ResourceWarning:

https://mail.python.org/pipermail/python-dev/2013-October/129923.html

I found a working solution using monkey patching of the socket, _pyio and builtins modules:

https://bitbucket.org/haypo/misc/src/tip/python/res_warn.py

This hack uses tracemalloc to retrieve the traceback where the file/socket was allocated, but you can do the same using traceback.extract_stack() for older Python version. So it's just an example to show how tracemalloc can be used.

Example with test_poplib (without res_warn.py):

[1/1] test_poplib /home/haypo/prog/python/default/Lib/test/support/init.py:1331: ResourceWarning: unclosed <ssl.SSLSocket fd=5, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 45747), raddr=('127.0.0.1', 48933)> gc.collect()

Ok, you now that the socket was destroyed be the garbage collector... But which test created this socket???

Example of res_warn.py with test_poplib:

[1/1] test_poplib /home/haypo/prog/python/default/Lib/test/support/init.py:1331: ResourceWarning: unclosed <ssl.SSLSocket fd=5, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 49715), raddr=('127.0.0.1', 43904)> gc.collect() Allocation traceback (most recent first): File '/home/haypo/prog/python/default/Lib/ssl.py', line 340 _context=self) File '/home/haypo/prog/python/default/Lib/poplib.py', line 390 self.sock = context.wrap_socket(self.sock) File '/home/haypo/prog/python/default/Lib/test/test_poplib.py', line 407 self.client.stls() File '/home/haypo/prog/python/default/Lib/unittest/case.py', line 567 self.setUp() File '/home/haypo/prog/python/default/Lib/unittest/case.py', line 610 return self.run(*args, **kwds) (...)

You can see that the socket was created by test_poplib at line 407 TestPOP3_TLSClass.setUp). It's more useful than the previous warning :-)

I found two issues in _pyio and tracemalloc modules. These issues should be fixed to use res_warn.py on text or buffered files and use it during Python shutdown:

http://bugs.python.org/issue19831 http://bugs.python.org/issue19829

Victor

More information about the Python-Dev mailing list