| msg27007 - (view) |
Author: ken668 (ken668) |
Date: 2005-12-09 01:03 |
| memory leak in email.message_from_string. This is what I did to create a leak. You used the attached file, memleak.eml. f = open("memleak.eml") buffer = f.read() f.close() # now buffer has the email string msg = email.message_from_string(buffer) msg = None # this should free the memory but it doesn't # The memory that is used in msg is not completely free |
|
|
| msg27008 - (view) |
Author: Neal Norwitz (nnorwitz) *  |
Date: 2005-12-11 20:13 |
| Logged In: YES user_id=33168 What causes you to believe this is a memory leak? I ran this under valgrind and it doesn't report any leaks. |
|
|
| msg27009 - (view) |
Author: ken668 (ken668) |
Date: 2005-12-12 15:53 |
| Logged In: YES user_id=1400763 I added these three lines after the line "msg=None" import gc print "gc.collect()\n\n", gc.collect() print "gc.garbage\n\n", gc.garbage If you pipe the output of gc.garbage to a file, you will see the email message you just sent are still be in the memory. Everytime I call email.message_from_string, a copy of the message will be kept in the memory even after I set the returned value to None. I tried the same thing with email package email-2.5.tar.gz, that memory was freed right after I set the variable "msg" to None. Thanks Ken |
|
|
| msg27010 - (view) |
Author: ken668 (ken668) |
Date: 2005-12-12 15:59 |
| Logged In: YES user_id=1400763 My mistake. I had also called gc.get_objects() too. It was gc.get_objects() that outputed the content of the email message. It was not gc.garbage. Sorry about that. |
|
|
| msg27011 - (view) |
Author: ken668 (ken668) |
Date: 2005-12-12 16:00 |
| Logged In: YES user_id=1400763 My mistake. I had also called gc.get_objects() too. It was gc.get_objects() that outputed the content of the email message. It was not gc.garbage. Sorry about that. |
|
|
| msg27012 - (view) |
Author: Neal Norwitz (nnorwitz) * |
Date: 2005-12-15 05:56 |
| Logged In: YES user_id=33168 Do you still believe there is a problem or can this report be closed? |
|
|
| msg27013 - (view) |
Author: ken668 (ken668) |
Date: 2005-12-15 06:44 |
| Logged In: YES user_id=1400763 Yes, it is a bug. My mistake was when I gave the details, I should have mentioned gc.get_objects(), not gc.garbage. gc.get_objects() showed the leaks not gc.garbage. |
|
|
| msg27014 - (view) |
Author: Fredrik Lundh (effbot) * |
Date: 2005-12-20 16:44 |
| Logged In: YES user_id=38376 This is not a leak (a leak means that the process will grow if you *repeat* the operation, not that things are unexpectedly left in the object memory). And it's not a bug, either; the "copy" of the message that's returned by get_objects() is the contents of the buffer variable in the test program. |
|
|
| msg27015 - (view) |
Author: ken668 (ken668) |
Date: 2005-12-20 18:37 |
| Logged In: YES user_id=1400763 May be you did not look at my test program more closely. Please look at the statement msg=None. The should removed all references of the object. The memory will definely grow if you repeat the operation. Why don't you run the email.message_from_string in a loop and see whether the memory grows. Thanks. |
|
|
| msg27016 - (view) |
Author: Fredrik Lundh (effbot) * |
Date: 2005-12-20 19:03 |
| Logged In: YES user_id=38376 Nope. If you do msg=None, you'll release a reference to the message object, and all direct, non-cyclic members. Any cyclic references held by the message object (or created by the email parser) will be removed at a later time, by the garbage collector. And even if you add an explicit gc.collect() call to your program, that won't remove the contents of the *buffer* variable. (If you run the program in a loop *without* an explicit gc.collect() call, the process will appear to grow for a while, since you're creating objects faster than the GC can remove them. Add gc.collect() to the loop, and the memory use will be rock solid.) |
|
|
| msg27017 - (view) |
Author: ken668 (ken668) |
Date: 2005-12-20 22:11 |
| Logged In: YES user_id=1400763 I assume the new email package has cyclic references in it. My program has garabage collection disabled because of performance issues. That means the email package will cause memory leak in my program. I will use the old package instead. Thanks for the help. |
|
|