[Python-Dev] C API for gc.enable() and gc.disable() (original) (raw)
Amaury Forgeot d'Arc amauryfa at gmail.com
Thu Sep 4 13:42:49 CEST 2008
- Previous message: [Python-Dev] C API for gc.enable() and gc.disable()
- Next message: [Python-Dev] Not releasing rc1 tonight
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hello,
Andrey Zhmoginov wrote:
I don't know if the following question is relevant, but it seems that many people here are familiar with Python cyclic garbage collector. I see Python [v2.5.2 (r252:60911, Jul 31 2008, 17:28:52)] crashing with Segment fault when I extend Python with a very simple module. This behavior is observed when I create a thousand of lists (it does not crash with 10-100) in the module with the garbage collector turned on. When I turn it off - everything is perfect. I suspect that it is my module, but if it is a Python bug ( GCMalloc? memory leak somewhere?), it may be worth reporting.
The gdb "where" reply is the following: #0 0x080d8de9 in PyErrOccurred () #1 0x080f508f in PyObjectGCMalloc () #2 0x080f5155 in PyObjectGCNew () #3 0x08079c98 in PyListNew () #4 0xb7f53519 in drawdsimple () from ./rt/rtcore.so #5 0xb7cf7833 in fficallSYSV () from /usr/lib/python2.5/lib-dynload/ctypes.so #6 0xb7cf766a in fficall () from /usr/lib/python2.5/lib-dynload/ctypes.so #7 0xb7cf2534 in CallProc () from /usr/lib/python2.5/lib-dynload/ctypes.so #8 0xb7cec02a in ?? () from /usr/lib/python2.5/lib-dynload/ctypes.so #9 0x0805cb97 in PyObjectCall () #10 0x080c7aa7 in PyEvalEvalFrameEx () #11 0x080c96e5 in PyEvalEvalFrameEx () #12 0x080cb1f7 in PyEvalEvalCodeEx () #13 0x080cb347 in PyEvalEvalCode () #14 0x080ea818 in PyRunFileExFlags () #15 0x080eaab9 in PyRunSimpleFileExFlags () #16 0x08059335 in PyMain () #17 0x080587f2 in main () The crashing python code is: from ctypes import * import gc core = CDLL( "./tst.so" ) core.a.argtypes = [] core.a.restype = pyobject #gc.disable() core.a() #gc.enable() And tst.cpp is: #include <Python.h> extern "C" { PyObject *a(); } PyObject *a() { int n = 1000; PyObject *item; for ( int i = 0; i < n; i++ ) item = PyListNew( 0 ); // Crashes here (somewhere in between). return item; } tst.cpp is compiled with: g++ -shared -Wl,-soname,tst.tmp.so -o tst.so tst.o g++ -I /usr/include/python2.5 -Wall -fPIC -o tst.o -c tst.cpp
This has nothing to do with the garbage collection, but with the GIL (the famous Global Interpreter Lock http://docs.python.org/api/threads.html )
When ctypes calls a CDLL function, it releases the GIL (to let eventual other threads run) Your example crashes because it calls python API functions when the GIL is not held, and is so invalid.
There are two solutions:
- re-acquire the GIL in your C functions with PyGILState_Ensure() & co
- use ctypes.PyDLL( "./tst.so" ), which does not release the GIL.
Hope this helps,
-- Amaury Forgeot d'Arc
- Previous message: [Python-Dev] C API for gc.enable() and gc.disable()
- Next message: [Python-Dev] Not releasing rc1 tonight
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]