| msg157876 - (view) |
Author: Aaron Meurer (Aaron.Meurer) |
Date: 2012-04-09 18:48 |
| Recently, after a small seemingly unrelated refactoring, the SymPy test suite in Python 3 started dying with "Fatal Python error: Cannot recover from stack overflow." Here's how to reproduce the error git clone git://github.com/sympy/sympy.git # Clone the development version of SymPy cd sympy git checkout 0856119bd7399a416c21e1692855a1077164f21c # This is the first commit to exhibit the problem. Do this in case we make an unrelated change that removes the problem. ./bin/use2to3 # Convert the codebase to Python 3 python3 py3k-sympy/setup.py test # Run the tests The issue is described in more detail at http://groups.google.com/group/sympy/browse_thread/thread/f664fe88e6b4f29d/3a44691c945695db#3a44691c945695db. Some key points: - The test that triggers the error is an XFAIL test (test that is expected to fail) that raises RuntimeError: maximum recursion depth exceeded. - The change that caused the error, 0856119bd7399a416c21e1692855a1077164f21c (see https://github.com/sympy/sympy/commit/0856119bd7399a416c21e1692855a1077164f21c), is seemingly unrelated. The only thing that I can think of is that it has added another call to the stack. - This kills Python with Abort Trap: 6 in Mac OS X, which generates a problem report to be sent to Apple. I have included a copy of it here: https://gist.github.com/2317869. - Others have reproduced this error as well, as can be seen by our test reporter tool. See the mailing list thread for more info. - I tested this in 3.2.3r2, and the error still occurred. I tried testing in the 3.3 alpha, but I could not get it to compile. |
|
|
| msg157915 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2012-04-09 22:36 |
| It looks like an issue in SymPy, a stack overflow. Why did you report this issue on CPython bug tracker? Is there an infinite loop in a recursive function call? Can you try to get the full Python traceback using faulthandler? Use "-X faulthandler" command line option, or use "import faulthandler; faulthandler.enable()" at the beginning of your program. You have to install it manually for Python < 3.3. |
|
|
| msg157917 - (view) |
Author: Aaron Meurer (Aaron.Meurer) |
Date: 2012-04-09 22:43 |
| We do have a stack overflow, but this should be raising a RuntimeError, not killing Python. The way it is now, Python dies completely with abort trap 6 (hence the Mac OS X problem report). Sorry if I didn't make this clear in the OP. |
|
|
| msg157918 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2012-04-09 22:48 |
| SymPy uses a C extension, numpy. The problem may be related to numpy. CPython tries to catch segmentation fault, but when a C extension is used, CPython may fail to protect your program against stack overflow. |
|
|
| msg157919 - (view) |
Author: Aaron Meurer (Aaron.Meurer) |
Date: 2012-04-09 22:50 |
| No it does not. SymPy is a pure Python library. |
|
|
| msg157923 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2012-04-09 23:27 |
| The (first) Python stack overflow occurs at: ----------- (gdb) py-bt Traceback (most recent call first): File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/expr.py", line 2531, in expand from sympy.simplify.simplify import fraction File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/cache.py", line 91, in wrapper func_cache_it_cache[k] = r = func(*args, **kw_args) File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 514, in as_real_imag return (self.expand(deep, **hints), S.Zero) File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 525, in _eval_expand_complex re_part, im_part = self.as_real_imag(deep=deep, **hints) File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/expr.py", line 2551, in expand expr = func(deep=deep, **hints) File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/cache.py", line 91, in wrapper func_cache_it_cache[k] = r = func(*args, **kw_args) File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 514, in as_real_imag return (self.expand(deep, **hints), S.Zero) File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 525, in _eval_expand_complex (gdb) where #0 _Py_CheckRecursiveCall (where=0x624f4b " in comparison") at Python/ceval.c:736 #1 0x0000000000419e6a in PyObject_RichCompare (v='sympy', w='sympy', op=2) at Objects/object.c:604 #2 0x0000000000419f2d in PyObject_RichCompareBool (v='sympy', w='sympy', op=2) at Objects/object.c:628 #3 0x00000000005e854e in lookdict (mp=0x7ffff1b3b1a8, key='sympy', hash=-191038920480525830) at Objects/dictobject.c:341 #4 0x00000000005e99ca in PyDict_GetItem (op= {'sympy.logic.inference': <module at remote 0x7fffeef80ba0>, ..., key='sympy') at Objects/dictobject.c:752 #5 0x00000000004f1510 in import_submodule (mod=None, subname='sympy', fullname='sympy') at Python/import.c:3313 #6 0x00000000004f064b in load_next (mod=None, altmod=None, inputname='sympy.simplify.simplify', p_outputname=0x7fffffe67e28, p_prefix=0x7fffffe67e20) at Python/import.c:3150 #7 0x00000000004ef610 in import_module_level (name='sympy.simplify.simplify', globals= {...}, locals=None, fromlist=('fraction',), level=0) at Python/import.c:2843 ----------- Ok, now I see the problem: import_submodule() uses PyDict_GetItem() whereas PyDict_GetItem() *clears* the exception (!). It should use PyDict_GetItemWithError() instead. Try attached patch. The problem is a generic problem: PyDict_GetItem() should be replaced by PyDict_GetItemWithError() everywhere... But it would be nice to start with import.c :-) |
|
|
| msg211163 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2014-02-13 19:13 |
| I didn't tested this patch with SymPy, but in general it LGTM. |
|
|
| msg229047 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2014-10-10 23:36 |
| This code fragment doesn't seem to exist any more. Since there's no test case, I'm going to close this. I presume it is no longer a problem for SymPy, since there's been no followup. |
|
|
| msg229049 - (view) |
Author: Aaron Meurer (Aaron.Meurer) |
Date: 2014-10-10 23:41 |
| The OP describes how to get the original code. Anyway, the issue was definitely fixed. |
|
|
| msg229050 - (view) |
Author: Aaron Meurer (Aaron.Meurer) |
Date: 2014-10-10 23:42 |
| Or do you mean the code in CPython doesn't exist any more? |
|
|
| msg229052 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2014-10-10 23:59 |
| Yes, it appears that the code does not exist in import.c any more. When I said "test case", I meant "unit test", sorry. |
|
|