[Python-Dev] New PEP: Using ssize_t as the index type (original) (raw)

M.-A. Lemburg mal at egenix.com
Tue Jan 10 10🔞58 CET 2006

Martin v. Löwis wrote:

Armin Rigo wrote:

This would do the right thing for <= 2.4, using ints everywhere; and the Python.h version 2.5 would detect the #define and assume it's a 2.5-compatible module, so it would override the #define with the real thing and turn on the ssizet interpretation of the '#' format character. This would be very similar to the PYSIZETCLEAN approach, except that it would also help to detect spelling mistakes.

From an implementation point of view, the real challenge is to give PyArgParseTuple a different meaning; I do this be #defining it to PyArgParseTupleSsizet (to preserve binary compatibility for the original interpretation of ParseTuple). Putting additional flags arguments in the entire code is also quite hackish. I still don't like the idea of a magic #define that changes the behavior of '#include <Python.h>', but I admit I don't find any better solution. I suppose I'll just blame C. More precisely, the printf style of function calling, and varargs functions. ISO C is pretty type safe, but with varargs functions, you lose that completely. I still hope I can write a C parser some day that does ParseTuple/BuildValue checking.

At least gcc does check the parameter types and generates warnings for wrong vararg parameters passed to printf() et al.

We definitely need a clean solution for this. Otherwise, authors who want to support more than just the latest Python release will run into deep trouble.

Note that the branch also has other changes of output parameter types which will cause problems in extensions (not only extensions implementing the sequence protocol as the PEP suggests, but also ones using it as well as extensions implementing or using the buffer protocol and the slicing protocol). These are not as easily avoidable as the PyArg_ParseTuple() problem which could be dealt with by a a new parser marker for ssize_t lengths (with the '#' marker casting the argument to an int).

I don't see a good solution for these other than introducing a set of new APIs (and maybe doing some macro magic as Martin did for PyArg_ParseTuple()). Due to the fact that changes in the types of output parameters require changes in the extension variable type layout itself, they introduce a large number of type changes in the extension and make writing backwards compatible extensions harder than necessary.

Furthermore, all extensions for Python 2.4 would have to be ported to the new Python API and porting is not going to be a simple recompile, but will require C skills.

We'd also have to make sure that old extensions don't just import with a warning, since the change will introduce buffer overflows and seg faults in extensions that are not aware of the change.

Martin, please add the above points to the PEP. I'd also like to see it published, because it's hard to track a PEP in the mailing

Thanks,

Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Jan 10 2006)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

More information about the Python-Dev mailing list