[Python-ideas] More details in MemoryError (original) (raw)

Charles-François Natali cf.natali at gmail.com
Tue Jan 22 08:13:23 CET 2013

2013/1/21 Oleg Broytman <phd at phdru.name>:

I'd very much like to see a situation when a program can survive MemoryError.

Let's say your using an image processing program. You have several images open on which you've been working for a couple minutes/hours. You open a new one, and it's so large that it results in MemoryError : instead of just losing all your current work (yeah, the program should support auto-save anyway, but let's pretend it doesn't), the program catches MemoryError, and displays a popup saying "No enough memory to process this image".

Now, sure, there are cases where an OOM condition will result in thrashing to death, or simply because of overcommit malloc() will never return NULL and you'll get nuked by the OOM killer, but depending on your operating system and allocation pattern, there are times when you can reasonably recover from a MemoryError. Also, a memory allocation failure doesn't necessarily mean you're OOM, it could be that youve exhausted your address space (on 32-bit), or hit RLIMIT_VM/RLIMIT_DATA.

2013/1/21 Benjamin Peterson <benjamin at python.org>:

What is this useful for?

Even if the exception isn't caught, if the extra information gets dumped in the traceback, it can be used for post-mortem debugging (to help distinguish between OOM, address space exhaustion, heap fragmentation, overflow in computation of malloc() argument, etc).

So I think it could probably be useful, but I see two problems:

So we can't easily capture the current allocated memory and the requested memory (the former could probably be retrieved in PyErr_NoMemory(), but the later would require modifying every call site and repeating it).

More information about the Python-ideas mailing list