[Python-Dev] Use of PyObject_NEW (original) (raw)

Guido van Rossum guido@python.org
Fri, 15 Mar 2002 11:09:51 -0500

I tried to understand the various memory allocation function and macros in Python, and found that there is probably a misunderstanding in what PyObjectNEW does.

For example, PyRangeNew says rangeobject *obj = PyObjectNEW(rangeobject, &PyRangeType); if (obj == NULL) return NULL; The assumption apparently is that somebody will raise a MemoryError and return NULL when allocation fails. However, this code expands to rangeobject obj = ((rangeobject)PyObjectInit( (PyObject *) malloc(((&PyRangeType)->tpbasicsize)), (&PyRangeType))); if (obj == ((void *)0) ) return ((void *)0) ; malloc will just return NULL in case of failure, and PyObjectInit starts with if (op == NULL) { PyErrSetString(PyExcSystemError, "NULL object passed to PyObjectInit"); return op; } So instead of a MemoryError, you will get a SystemError if the system runs out of memory. Is that intentional?

Oh, good catch. PyObject_Init() should only raise its own exception when PyErr_Occurred() returns NULL.

The documentation says

Macro version of \cfunction{PyObjectNew()}, to gain performance at the expense of safety. This does not check \var{type} for a \NULL{} value. This is incorrect: It does check for NULL.

It checks the allocated storage for NULL, but it doesn't check the 'type' argument for NULL. Of course it doesn't! The function doesn't either. The type is always the address of a statically allocated variable so there's no worry about NULL here. So there's something else wrong with the docs -- why does it say this at all?

It also does not help to gain performance - PyObjectNew has three calls (PyObjectNew, malloc, PyNewReference), and so does PyObjectNEW (malloc, PyObjectInit, PyNewReference).

Curious. You seem to be right. I'm pretty sure it used to be faster to use the macros, but I believe correctness requirements have made the point moot.

I recommend to deprecate PyObjectNEW (and correspondingly PyObjectNEWVAR, PyObjectDEL).

And in the mean time, define them as aliases for the functions -- that should save some code size.

--Guido van Rossum (home page: http://www.python.org/~guido/)