Issue 5437: Singleton MemoryError can hold traceback data and locals indefinitely (original) (raw)

Created on 2009-03-08 01:19 by pitrou, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
memerr.py pitrou,2009-03-08 01:19
memerror.patch pitrou,2009-10-29 21:22
memerror.patch pitrou,2010-10-28 17:41
Messages (10)
msg83292 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-08 01:19
The PyExc_MemoryErrorInst object is persistent and its members never get cleared. This means any local variable which gets caught in the traceback isn't deallocated until the next MemoryError (!). Sample script demonstrates this. (this doesn't seem to affect 2.x because the traceback isn't attached to the exception instance)
msg83293 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-08 01:21
The following patch fixes the case when the exception is discarded in Python, but not when e.g. PyErr_Clear() is used from C code.
msg83296 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-08 01:34
A proper fix would probably be to maintain a bunch of preallocated instances in a freelist rather than relying on an explicit singleton. It would enforce proper dereferencing and garbage collection semantics.
msg89181 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2009-06-10 02:40
Honestly, I don't think it is a big issue. MemoryErrors are rare and typically cause the interpreter to shutdown. By the way, do you think the static PyExc_RecursionErrorInst object is affected by this bug?
msg94684 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-10-29 21:22
Here is a patch preallocating a freelist of MemoryErrors. To Alexandre: yes, PyExc_RecursionErrorInst probably has the same problem.
msg110325 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-07-14 20:31
Would someone please check out thie patch, it's mostly changes to C code that I'm not qualified to comment on.
msg112001 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-07-29 20:41
Changing priority as nobody appears to have lost any sleep over this issue.
msg119806 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-28 17:41
Updated patch against py3k.
msg119822 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-28 22:41
I can't manage to reproduce the issue with PyExc_RecursionErrorInst. It seems this instance is only used in a very select condition, that is, when the recursion limit is hit when trying to "normalize" an exception. Therefore, I will add a test without "fixing" anything.
msg119823 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-28 23:07
Patch with improved tests committed in r85896 (3.2) and r85898 (3.1).
History
Date User Action Args
2022-04-11 14:56:46 admin set github: 49687
2010-10-28 23:07:25 pitrou set status: open -> closedresolution: fixedmessages: + stage: patch review -> resolved
2010-10-28 22:41:15 pitrou set messages: +
2010-10-28 17:41:33 pitrou set files: + memerror.patchmessages: +
2010-07-29 20:41:41 BreamoreBoy set priority: high -> normalmessages: +
2010-07-14 20:31:35 BreamoreBoy set nosy: + BreamoreBoymessages: +
2009-10-29 21:23:10 pitrou set stage: needs patch -> patch review
2009-10-29 21:23:01 pitrou set files: - issue5437.patch
2009-10-29 21:22:44 pitrou set files: + memerror.patchmessages: + versions: + Python 3.2, - Python 3.0
2009-06-10 02:40:46 alexandre.vassalotti set nosy: + alexandre.vassalottimessages: +
2009-03-08 01:34:57 pitrou set messages: +
2009-03-08 01:21:56 pitrou set files: + issue5437.patchkeywords: + patchmessages: +
2009-03-08 01:19:50 pitrou create