[Python-Dev] PyObject_New vs PyObject_NEW (original) (raw)

David Abrahams dave@boost-consulting.com
Wed, 12 Mar 2003 13:03:35 -0500

Tim Peters <tim.one@comcast.net> writes:

[Guido]

You can read the source code as well as I can. Possibly, but not as well as I can -- the memory API's implementation is monumentally convoluted, especially before 2.3. Speaking of which, David, which version of Python was "someone" using?

I was the one who discovered the problem, using Python 2.2.2. Curiously, "someone" missed it because he was using vc6 instead of vc7.

Did they enable pymalloc?

I don't think I did that. I don't exactly know what pymalloc is.

Did they give you a traceback (showing from where free() was called)?

I can get one for you. Here:

MSVCRTD.DLL!freedbglk(void * pUserData=0x00c46338, int nBlockUse=1) Line 1044 + 0x30 C MSVCRTD.DLL!_free_dbg(void * pUserData=0x00c46338, int nBlockUse=1) Line 1001 + 0xd C MSVCRTD.DLL!free(void * pUserData=0x00c46338) Line 956 + 0xb C python22_d.dll!_PyObject_Del(_object * op=0x00c46338) Line 146 + 0xa C opaque_ext_d.pyd!dealloc(_object * self=0x00c46338) Line 12 + 0xa C++ python22_d.dll!_Py_Dealloc(_object * op=0x00c46338) Line 1837 + 0x7 C python22_d.dll!tupledealloc(PyTupleObject * op=0x0093c9a8) Line 147 + 0x70 C python22_d.dll!_Py_Dealloc(_object * op=0x0093c9a8) Line 1837 + 0x7 C python22_d.dll!do_call(_object * func=0x00c45e00, _object * * * pp_stack=0x0012edf8, int na=1, int nk=0) Line 3273 + 0x43 C python22_d.dll!eval_frame(_frame * f=0x008c2e68) Line 2038 + 0x1e C python22_d.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x00963068, _object * globals=0x008e65a8, _object * locals=0x008e65a8, _object * * args=0x00000000, int argcount=0, _object * * kws=0x00000000, int kwcount=0, _object * * defs=0x00000000, int defcount=0, _object * closure=0x00000000) Line 2595 + 0x9 C python22_d.dll!PyEval_EvalCode(PyCodeObject * co=0x00963068, _object * globals=0x008e65a8, _object * locals=0x008e65a8) Line 486 + 0x1f C python22_d.dll!exec_statement(_frame * f=0x008efbf0, _object * prog=0x00963068, _object * globals=0x008e65a8, _object * locals=0x008e65a8) Line 3668 + 0x11 C python22_d.dll!eval_frame(_frame * f=0x008efbf0) Line 1482 + 0x15 C python22_d.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x008c79a8, _object * globals=0x008e5940, _object * locals=0x00000000, _object * * args=0x00965a7c, int argcount=7, _object * * kws=0x00965a98, int kwcount=0, _object * * defs=0x00000000, int defcount=0, _object * closure=0x00000000) Line 2595 + 0x9 C python22_d.dll!fast_function(_object * func=0x0095cc98, _object * * * pp_stack=0x0012f268, int n=7, int na=7, int nk=0) Line 3173 + 0x41 C python22_d.dll!eval_frame(_frame * f=0x00965900) Line 2035 + 0x25 C python22_d.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x008c6fd8, _object * globals=0x008e5940, _object * locals=0x00000000, _object * * args=0x008d8c64, int argcount=5, _object * * kws=0x008d8c78, int kwcount=0, _object * * defs=0x00000000, int defcount=0, _object * closure=0x00000000) Line 2595 + 0x9 C python22_d.dll!fast_function(_object * func=0x009551a8, _object * * * pp_stack=0x0012f494, int n=5, int na=5, int nk=0) Line 3173 + 0x41 C python22_d.dll!eval_frame(_frame * f=0x008d8af0) Line 2035 + 0x25 C python22_d.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x008cee50, _object * globals=0x008e5940, _object * locals=0x00000000, _object * * args=0x008c75d8, int argcount=5, _object * * kws=0x008c75ec, int kwcount=0, _object * * defs=0x0092d7a4, int defcount=3, _object * closure=0x00000000) Line 2595 + 0x9 C python22_d.dll!fast_function(_object * func=0x00955220, _object * * * pp_stack=0x0012f6c0, int n=5, int na=5, int nk=0) Line 3173 + 0x41 C python22_d.dll!eval_frame(_frame * f=0x008c7450) Line 2035 + 0x25 C python22_d.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x008ddba0, _object * globals=0x008e5940, _object * locals=0x00000000, _object * * args=0x0089582c, int argcount=3, _object * * kws=0x00895838, int kwcount=0, _object * * defs=0x0092e2cc, int defcount=1, _object * closure=0x00000000) Line 2595 + 0x9 C python22_d.dll!fast_function(_object * func=0x0095a850, _object * * * pp_stack=0x0012f8ec, int n=3, int na=3, int nk=0) Line 3173 + 0x41 C python22_d.dll!eval_frame(_frame * f=0x008956a8) Line 2035 + 0x25 C python22_d.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x008e3848, _object * globals=0x008e5940, _object * locals=0x00000000, _object * * args=0x008787f4, int argcount=1, _object * * kws=0x008787f8, int kwcount=0, _object * * defs=0x0092eaa4, int defcount=5, _object * closure=0x00000000) Line 2595 + 0x9 C python22_d.dll!fast_function(_object * func=0x0095e620, _object * * * pp_stack=0x0012fb18, int n=1, int na=1, int nk=0) Line 3173 + 0x41 C python22_d.dll!eval_frame(_frame * f=0x00878690) Line 2035 + 0x25 C python22_d.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x008ce618, _object * globals=0x0086f780, _object * locals=0x00000000, _object * * args=0x008777bc, int argcount=0, _object * * kws=0x008777bc, int kwcount=0, _object * * defs=0x0084e56c, int defcount=1, _object * closure=0x00000000) Line 2595 + 0x9 C python22_d.dll!fast_function(_object * func=0x008736e8, _object * * * pp_stack=0x0012fd44, int n=0, int na=0, int nk=0) Line 3173 + 0x41 C python22_d.dll!eval_frame(_frame * f=0x00877660) Line 2035 + 0x25 C python22_d.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x0084d8a8, _object * globals=0x0086f780, _object * locals=0x0086f780, _object * * args=0x00000000, int argcount=0, _object * * kws=0x00000000, int kwcount=0, _object * * defs=0x00000000, int defcount=0, _object * closure=0x00000000) Line 2595 + 0x9 C python22_d.dll!PyEval_EvalCode(PyCodeObject * co=0x0084d8a8, _object * globals=0x0086f780, _object * locals=0x0086f780) Line 486 + 0x1f C python22_d.dll!run_node(_node * n=0x0088e730, char * filename=0x00842def, _object * globals=0x0086f780, _object * locals=0x0086f780, PyCompilerFlags * flags=0x0012ff38) Line 1079 + 0x11 C python22_d.dll!run_err_node(_node * n=0x0088e730, char * filename=0x00842def, _object * globals=0x0086f780, _object * locals=0x0086f780, PyCompilerFlags * flags=0x0012ff38) Line 1066 + 0x19 C python22_d.dll!PyRun_FileExFlags(_iobuf * fp=0x10261888, char * filename=0x00842def, int start=257, _object * globals=0x0086f780, _object * locals=0x0086f780, int closeit=1, PyCompilerFlags * flags=0x0012ff38) Line 1057 + 0x19 C python22_d.dll!PyRun_SimpleFileExFlags(_iobuf * fp=0x10261888, char * filename=0x00842def, int closeit=1, PyCompilerFlags * flags=0x0012ff38) Line 686 + 0x22 C python22_d.dll!PyRun_AnyFileExFlags(_iobuf * fp=0x10261888, char * filename=0x00842def, int closeit=1, PyCompilerFlags * flags=0x0012ff38) Line 495 + 0x15 C python22_d.dll!Py_Main(int argc=2, char * * argv=0x00842db8) Line 367 + 0x30 C python_d.exe!main(int argc=2, char * * argv=0x00842db8) Line 10 + 0xd C python_d.exe!mainCRTStartup() Line 338 + 0x11 C kernel32.dll!77e814c7()

Was it even freeing a Python object at the time?

Yup.

In what code base did someone make this substitution (e.g., Python core, Boost sources, someone's own extension module, someone else's extension module)?

Boost sources

The straight answer to your question is no. A nastier answer is that many memory mgmt screwups are shy, and can be triggered by seemingly irrelevant changes.

Both answers seem to amount to "'someone' must have a bug in his code". Am I reading that correctly?

-- Dave Abrahams Boost Consulting www.boost-consulting.com