[Python-Dev] Use of PyObject_NEW (original) (raw)

Tim Peters tim.one@comcast.net
Fri, 15 Mar 2002 03:29:06 -0500

[Martin v. Loewis]

I tried to understand the various memory allocation function and macros in Python, and found that there is probably a misunderstanding in what PyObjectNEW does.

For example, PyRangeNew says rangeobject *obj = PyObjectNEW(rangeobject, &PyRangeType); if (obj == NULL) return NULL; The assumption apparently is that somebody will raise a MemoryError and return NULL when allocation fails. However, this code expands to rangeobject obj = ((rangeobject)PyObjectInit( (PyObject *) malloc(((&PyRangeType)->tpbasicsize)), (&PyRangeType))); if (obj == ((void *)0) ) return ((void *)0) ; malloc will just return NULL in case of failure, and PyObjectInit starts with if (op == NULL) { PyErrSetString(PyExcSystemError, "NULL object passed to PyObjectInit"); return op; } So instead of a MemoryError, you will get a SystemError if the system runs out of memory. Is that intentional?

Not credible -- it's a bug. PyObject_NEW and PyObject_New should behave alike, and PyObject_New does set MemoryError in this case (via _PyObject_New().

The documentation says

Macro version of \cfunction{PyObjectNew()}, to gain performance at the expense of safety. This does not check \var{type} for a \NULL{} value. This is incorrect: It does check for NULL.

No, it's correct: as stated, it does not check \var{type} for a NULL. Note that the docs name the second argument "type", while the macro names the first argument "type". Neither PyObject_New nor PyObject_NEW check their second argument for NULL (so it's a minor mystery too why the docs bother to point this out only for PyObject_NEW).

It also does not help to gain performance - PyObjectNew has three calls (PyObjectNew, malloc, PyNewReference), and so does PyObjectNEW (malloc, PyObjectInit, PyNewReference).

I recommend to deprecate PyObjectNEW (and correspondingly PyObjectNEWVAR, PyObjectDEL).

I think that's pointless -- lots of code uses these things, and we have no effective way to deprecate pieces of the C API anyway. I'd be in favor of redefining

#define PyObject_NEW PyObject_New

to get rid of the MemoryError bug, though.