| msg74060 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2008-09-30 01:10 |
| I would like to be able to catch SIGSEGV in my Python code! So I started to hack Python trunk to support this feature. The idea is to use a signal handler which call longjmp(), and add setjmp() at Py_EvalFrameEx() enter. See attached ("small") patch: segfault.patch Example read.py with the *evil* ctypes module of invalid memory read: ------------------- 8< -------------- from ctypes import string_at def fault(): text = string_at(1, 10) print("text = {0!r}".format(text)) def test(): print("test: 1") try: fault() except MemoryError, err: print "ooops!" print err print("test: 2") try: fault() except MemoryError, err: print "ooops!" print err print("test: end") def main(): test() if __name__ == "__main__": main() ------------------- 8< -------------- Result: ------------------- 8< -------------- $ python read.py test: 1 sizeof()=160 ooops! segmentation fault test: 2 sizeof()=160 ooops! segmentation fault test: end ------------------- 8< -------------- Example bug1.py of a stack overflow: ---------- loop = None, for i in xrange(10**5): loop = loop, None ---------- Result: ---------- $ python -i bug1.py (((((((((...Traceback (most recent call last): File "", line 1, in MemoryError: segmentation fault ---------- Python is able to restore a valid state (stack/heap) after a segmentation fault and raise a classical Python exception (I choosed MemoryError, but it could be a specific exception). On my computer (Ubuntu Gutsy/i386), each segfault_frame takes sizeof(sigjmpbuf) + sizeof(void*) = 160 bytes, allocated on the stack. I don't know if it's huge or not, but that will limit the number of recursive calls. The feature can be optional if we add a configure option and some #ifdef/#endif. A dedicated stack is needed to be call the signal handler on stack overflow error. I choosed 4 KB, but since I only call longjmp(), smaller stack might also works. Does other VM support such feature? JVM, Mono, .NET, etc. ? I had the idea of catching SIGSEGV after reading the issue 1069092 (stack overflow because of too many recursive calls). |
|
|
| msg74066 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2008-09-30 08:54 |
| Did you consider using PyOS_CheckStack for this? Currently there is only a Windows implementation, but it seems that the primitives you use in your patch could form a Unix version. |
|
|
| msg74067 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2008-09-30 09:02 |
| @amaury.forgeotdarc: It looks like PyOS_CheckStack() is only implemented for Windows. It uses alloca() + __try/__except + _resetstkoflw(). The GNU libc nor Linux kernel don't check stack pointer on alloca(), it's just $esp += . Using alloca() you may also be able to able outside the stack to move your stack pointer to the heap or another memory mapping. PyOS_CheckStack() doesn't really protect the stack: if a function use alloca() or a similar construction like « void test(int size) { char allocated_on_the_stack[size]; ... } », you will not catch this error. PyOS_CheckStack() only checks one type of error: stack overflow. It doesn't check invalid memory read / write (see my first example, read.py). |
|
|
| msg74068 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2008-09-30 09:06 |
| Note: my patch can be adapted to catch SIGFPE (divison by zero or other math error). For int/long types, Python avoids divison by zero, but for code written in C ("external modules"), Python is unable to catch such errors. Eg. see last imageop issue: i was possible to generate many divisions by zero. |
|
|
| msg74089 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2008-09-30 18:12 |
| Oops, my patch was broken. I forgot to install the fault handler! Here is a new version of the patch which also catch SIGFPE: raise an ArithmeticError. |
|
|
| msg77478 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2008-12-10 02:50 |
| New patch: - limit memory footprint: use a static buffer to store the frames, with a maximum of MAXDEPTH frames (default: MAXDEPTH=100) - if there are more than MAXDEPTH frames, jump to the frame MAXDEPTH on error (it's like a truncated traceback) - don't call segfault_exit() in PyEval_EvalFrameEx() if segfault_enter() was not called On Ubuntu Gutsy, for MAXDEPTH=100 the memory footprint is 19996 bytes. I think that it's small, but it's possible to reduce the footprint by using less frames or disable the alternative stack (needed for stack overflow errors). |
|
|
| msg77479 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2008-12-10 02:54 |
| Oh, another change in segfault-3.patch: - disable signal handler before the first call to segfault_enter() and the last call to segfault_exit() About the memory footprint: it would be possible to use variable size buffer using malloc() and then realloc(). But static buffers are easier to use, and I don't want to play with malloc() while I'm handling a segmentation fault or stack overflow :-) |
|
|
| msg77563 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2008-12-10 18:34 |
| fault.py: catch two segfaults in Python and C contexts. |
|
|
| msg78745 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2009-01-02 01:30 |
| As mentioned in python-dev, the patch would be more suitable for inclusion if it was changed to simply print a stack trace and bail out, rather than try to resume execution of the Python program. |
|
|
| msg95055 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2009-11-09 01:43 |
| My idea was rejected on python-dev mailing list. But I'm unable to write a patch to dump a backtrace on segfault. Anyway it would be a complelty different patch (and so a different issue). So I prefer to close this (old) issue. |
|
|
| msg95095 - (view) |
Author: Adam Olsen (Rhamphoryncus) |
Date: 2009-11-09 18:51 |
| That's fine, but please provide a link to the new issue once you create it. |
|
|
| msg106804 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2010-05-31 19:16 |
| > That's fine, but please provide a link to the new issue once you create it. Done: issue #8863. |
|
|