Issue 20967: hashlib memory leak (original) (raw)

Created on 2014-03-18 14:03 by ateng, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (5)

msg213961 - (view)

Author: Adrian Teng (ateng)

Date: 2014-03-18 14:03

A particular usage pattern of hashlib will cause a memory leak.

This leaks: import hashlib import sys

if name == 'main': data_sha1 = "hello world" data_md5 = "hello world" for i in xrange(int(1e6)): hashlib.sha1(data_sha1) hashlib.md5(data_md5)

    if i % 1000 == 0:
        print sys.getrefcount(data_sha1), ",", sys.getrefcount(data_md5)

this doesn't leak:

import hashlib import sys

if name == 'main': data_sha1 = "hello world" data_md5 = "hello world" for i in xrange(int(1e6)): sha1 = hashlib.sha1() sha1.update(data_sha1) md5 = hashlib.md5() md5.update(data_md5)

    if i % 1000 == 0:
        print sys.getrefcount(data_sha1), ", ", sys.getrefcount(data_md5)

See attached for leak memory profiling in linux

msg213977 - (view)

Author: STINNER Victor (vstinner) *

Date: 2014-03-18 15:05

I'm unable to reproduce your issue with Python 2.7.5:

$ python memoryleak_min.py [256720896.0, 15740928.0, 139264.0] [256724992.0, 15962112.0, 139264.0] [256724992.0, 15966208.0, 139264.0] [256724992.0, 15966208.0, 139264.0] (...) [256724992.0, 15966208.0, 139264.0]

$ python2.7 5 , 5 5 , 5 (...) 5 , 5

What is your exact Python version? What is your OS? OS version?

msg213978 - (view)

Author: Adrian Teng (ateng)

Date: 2014-03-18 15:14

Python 2.7.3, Red Hat Enterprise Linux Server release 5.5, with kernal 2.6.18-308.el5

msg213981 - (view)

Author: STINNER Victor (vstinner) *

Date: 2014-03-18 15:41

"Python 2.7.3, Red Hat Enterprise Linux Server release 5.5, with kernal 2.6.18-308.el5"

So your version is older than mine. Let me check Misc/NEWS... Oh...

"Issue #15219: Fix a reference leak when hashlib.new() is called with invalid parameters."

in the "What's New in Python 2.7.4 release candidate 1" section.

Can you please try a more recent Python 2.7 version to check if it's the same bug?

msg214094 - (view)

Author: Adrian Teng (ateng)

Date: 2014-03-19 14:17

Yup. Tested on 2.7.5 and it doesn't leak. I guess this is a duplicate of #15219. Cheers!

History

Date

User

Action

Args

2022-04-11 14:58:00

admin

set

github: 65166

2014-03-19 14🔞12

brett.cannon

set

status: open -> closed

2014-03-19 14:17:12

ateng

set

resolution: duplicate
messages: +

2014-03-18 18:01:18

jcea

set

nosy: + jcea

2014-03-18 15:41:32

vstinner

set

messages: +

2014-03-18 15:14:19

ateng

set

messages: +

2014-03-18 15:05:45

vstinner

set

nosy: + vstinner
messages: +

2014-03-18 14:03:55

ateng

create