Issue 33526: hashlib leak on import (original) (raw)

Issue33526

Created on 2018-05-15 20:49 by thehesiod, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (3)
msg316723 - (view)	Author: Alexander Mohr (thehesiod) *	Date: 2018-05-15 20:49
I'm seeing a lot of leaks via valgrind against the hashlib module. It appears that it's calling OpenSSL_add_all_digests(); on init, and never calling the corresponding EVP_Cleanup on free: https://www.openssl.org/docs/man1.1.0/crypto/OpenSSL_add_all_digests.html. I see a ton of leaks like the following: ==27765== 24 bytes in 1 blocks are still reachable in loss record 13 of 10,294 ==27765== at 0x4C28C7B: malloc (vg_replace_malloc.c:299) ==27765== by 0xA92E337: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.2k) ==27765== by 0xA9E325A: lh_insert (in /usr/lib64/libcrypto.so.1.0.2k) ==27765== by 0xA93103E: OBJ_NAME_add (in /usr/lib64/libcrypto.so.1.0.2k) ==27765== by 0xA9F3559: OpenSSL_add_all_digests (in /usr/lib64/libcrypto.so.1.0.2k) ==27765== by 0xA44CF02: PyInit__hashlib (_hashopenssl.c:998) ==27765== by 0x506E627: _PyImport_LoadDynamicModuleWithSpec (importdl.c:154) ==27765== by 0x506DBA7: _imp_create_dynamic_impl (import.c:2008) ==27765== by 0x5067A2A: _imp_create_dynamic (import.c.h:289) ==27765== by 0x4F3061A: PyCFunction_Call (methodobject.c:114) ==27765== by 0x503E10C: do_call_core (ceval.c:5074) ==27765== by 0x5035F30: _PyEval_EvalFrameDefault (ceval.c:3377) ==27765== by 0x502280F: PyEval_EvalFrameEx (ceval.c:718) ==27765== by 0x503A944: _PyEval_EvalCodeWithName (ceval.c:4139) ==27765== by 0x503DA4D: fast_function (ceval.c:4950) ==27765== by 0x503D3FC: call_function (ceval.c:4830) ==27765== by 0x5035563: _PyEval_EvalFrameDefault (ceval.c:3295) ==27765== by 0x502280F: PyEval_EvalFrameEx (ceval.c:718) ==27765== by 0x503D70D: _PyFunction_FastCall (ceval.c:4891) ==27765== by 0x503D922: fast_function (ceval.c:4926) ==27765== by 0x503D3FC: call_function (ceval.c:4830) ==27765== by 0x5035563: _PyEval_EvalFrameDefault (ceval.c:3295) ==27765== by 0x502280F: PyEval_EvalFrameEx (ceval.c:718) ==27765== by 0x503D70D: _PyFunction_FastCall (ceval.c:4891) ==27765== by 0x503D922: fast_function (ceval.c:4926) ==27765== by 0x503D3FC: call_function (ceval.c:4830) ==27765== by 0x5035563: _PyEval_EvalFrameDefault (ceval.c:3295) ==27765== by 0x502280F: PyEval_EvalFrameEx (ceval.c:718) ==27765== by 0x503D70D: _PyFunction_FastCall (ceval.c:4891) ==27765== by 0x503D922: fast_function (ceval.c:4926) ==27765== by 0x503D3FC: call_function (ceval.c:4830) ==27765== by 0x5035563: _PyEval_EvalFrameDefault (ceval.c:3295) ==27765== by 0x502280F: PyEval_EvalFrameEx (ceval.c:718) ==27765== by 0x503D70D: _PyFunction_FastCall (ceval.c:4891) ==27765== by 0x503D922: fast_function (ceval.c:4926) I'm not exactly sure how this is happening yet (I know the code I use does a __import__ and uses multiple threads). It sounds like this call should be ref-counted or perhaps only done once for the life of the application.
msg316737 - (view)	Author: Christian Heimes (christian.heimes) *	Date: 2018-05-15 21:47
Thanks for notifying us. There isn't much we can do here. Python doesn't support unloading of C extension modules. The state will persist until the process ends. It doesn't make much sense to call EVP_cleanup() on shutdown, too. It will just slow down shutdown. The memory is freed anyway once the process has terminated.
msg316738 - (view)	Author: Alexander Mohr (thehesiod) *	Date: 2018-05-15 21:51
closing as I'm not quite sure this is right