[Python-Dev] Let's update CObject API so it is safe and regular! (original) (raw)

Guido van Rossum guido at python.org
Thu Apr 2 01:53:28 CEST 2009

2009/4/1 Larry Hastings <larry at hastings.org>:

Jim Fulton wrote: The only type-safety mechanism for a CObject is it's identity.  If you want to make sure you're using the foomodule api, make sure the address of the CObject is the same as the address of the api object exported by the module. That doesn't help.  Here's a program that crashes the interpreter, something I shouldn't be able to do from pure Python: import socket import cStringIO cStringIO.cStringIOCAPI = socket.CAPI import cPickle s = cPickle.dumps([1, 2, 3]) How can cPickle determine that cStringIO.cStringIOCAPI is legitimate?

This is a bug in cPickle. It calls the PycString_IMPORT macro at the very end of its init_stuff() function without checking for success. This macro calls PyCObject_Import("cStringIO", "cStringIO_CAPI") which in turn calls PyCObject_AsVoidPtr() on the object that it finds as cStringIO.cStringIO_CAPI, and this function does do a type check and sets an exception if the object isn't a PyCObject instance. However cPickle's initialization doesn't check for errors immediately and apparently some later code overrides the exception.

The fix should be simple: insert

if (PyErr_Occurred()) return -1;

immediately after the line

PycString_IMPORT;

in init_stuff() in cPickle.c. This will cause the import of cPickle to fail with an exception and all should be well.

I have to say, I haven't understood this whole thread, but I'm skeptical about a redesign. But perhaps you can come up with an example that doesn't rely on this cPickle bug?

--Guido

That would break backward compatibility. Are you proposing this for Python 3?

I'm proposing this for Python 3.1.  My understanding is that breaking backwards compatibility is still on the table, which is why I wrote the patch the way I did.  If we have to preserve the existing API, I still think we should add new APIs and deprecate the old ones. It's worth noting that there's been demand for this for a long time.  Check out this comment from Include/datetime.h: _#define PyDateTimeIMPORT _ _PyDateTimeAPI = (PyDateTimeCAPI*) PyCObjectImport("datetime", _ "datetimeCAPI") /* This macro would be used if PyCObjectImportEx() was created. _#define PyDateTimeIMPORT _ _PyDateTimeAPI = (PyDateTimeCAPI*) PyCObjectImportEx("datetime", _ "datetimeCAPI", __ DATETIMEAPIMAGIC) */ That was checked in by Tim Peters on 2004-06-20, r36214.  (At least, in the py3k/trunk branch; I'd hope it would be the same revision number in other branches.)

/larry/

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list