[Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 (original) (raw)

M.-A. Lemburg mal at egenix.com
Mon Jun 2 14:33:08 CEST 2008

On 2008-06-02 01:30, Gregory P. Smith wrote:

On Fri, May 30, 2008 at 1:37 AM, M.-A. Lemburg <mal at egenix.com> wrote:

Sorry, I probably wasn't clear enough:

Why can't we have both PyString and PyBytes exposed as C APIs (ie. visible in code and in the linker) in 2.x, with one redirecting to the other ?

* Why should the 2.x code base turn to hacks, just because 3.x wants to restructure itself ? With the better explanation from Greg of what the checked in approach achieves (i.e. preserving exact ABI compatibility for PyString*, while allowing PyBytes* to be used at the source code level), I don't see what has been done as being any more of a hack than the possibly more common "#define " (which would break binary compatibility). The only things that I think would tidy it up further would be to: - include an explanation of the approach and its effects on API and ABI backward and forward compatibility within 2.x and between 2.x and 3.x in stringobject.h - expose the PyBytes* functions to the linker in 2.6 as well as 3.0 Which is what I was suggesting all along; sorry if I wasn't clear enough on that. The standard approach is that you provide #define redirects from the old APIs to the new ones (which are then picked up by the compiler) and add function wrappers to the same affect (to make linkers, dynamic load APIs such ctypes and debuggers happy). Example from pythonrun.h|c: --------------------------- /* Use macros for a bunch of old variants */ #define PyRunString(str, s, g, l) PyRunStringFlags(str, s, g, l, NULL) /* Deprecated C API functions still provided for binary compatiblity */ #undef PyRunString PyAPIFUNC(PyObject *) PyRunString(const char *str, int s, PyObject *g, PyObject *l) { return PyRunStringFlags(str, s, g, l, NULL); } Okay, how about this? http://codereview.appspot.com/1521 Using that patch, both PyString and PyBytes APIs are available using function stubs similar to the above. I opted to define the stub functions right next to the ones they were stubbing rather than putting them all at the end of the file or in another file but they could be moved if someone doesn't like them that way.

Thanks. I was working on a similar patch. Looks like you beat me to it.

The only thing I'm not sure about is having the wrappers in the same file - this is likely to cause merge conflicts when doing direct merging and even with an automated renaming approach, the extra code would be easier to remove if it were e.g. at the end of the file or even better: in a separate file.

My patch worked slightly differently: it adds wrappers PyString* that forward calls to the PyBytes* APIs and they all live in stringobject.c. stringobject.h then also provides aliases so that recompiled extensions pick up the new API names.

While working on my patch I ran into an issue that I haven't been able to resolve: the wrapper functions got optimized away by the linker and even though they appear in the libpython2.6.a, they don't end up in the python binary itself.

As a result, importing Python 2.5 in the resulting 2.6 binary still fails with a unresolved PyString symbol.

Please check whether that's the case for your patch as well.

I still believe that we should not make "easy of merging" the primary motivation for backporting changes in 3.x to 2.x. Software design should not be guided by restrictions in the tool chain, if not absolutely necessary.

The main argument for a backport needs to be general usefulness to the 2.x users, IMHO... just like any other feature that makes it into 2.x. If merging is difficult then this needs to be addressed, but there are more options to that than always going back to the original 2.x trunk code. I've given a few suggestions on how this could be approached in other emails on this thread. I am not the one doing the merging or working on merge tools so I'll leave this up to those that are.

I'm not sure whether there are any specific merge tools around - apart from the 2to3.py script.

There also doesn't seem to be any documentation on the merge process itself (at least nothing that Google can find in the PEPs), so it's difficult to make any suggestions.

Thanks,

Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Jun 02 2008)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

2008-07-07: EuroPython 2008, Vilnius, Lithuania 34 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::

eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
        Registered at Amtsgericht Duesseldorf: HRB 46611

More information about the Python-Dev mailing list