[Python-Dev] PEP 384: Defining a Stable ABI (original) (raw)

M.-A. Lemburg mal at egenix.com
Mon May 25 19:41:54 CEST 2009

Martin v. Löwis wrote:

Thomas Wouters reminded me of a long-standing idea; I finally found the time to write it down.

Please comment! ...

Up until this PEP proposal, we had a very simple scheme for the Python C-API: all documented functions and variables with a "Py" prefix were part of the C-API, everything else was not and could change between releases (in particular the private "_Py" prefix APIs).

Changing the published APIs was considered a bad thing in the 2.x development process and generally required a good reason to get supported.

Changing private functions or ones that were not documented was generally never a big problem.

Now, with the PEP, I have a feeling that the Python C-API will in effect be limited to what's in the PEP's idea of a usable ABI and open up the non-inluded public C-APIs to the same rate of change as the private APIs.

If that's the case, the PEP should be discussed on the C-API list first, in order to identify a complete list of APIs that is supposed to define the Python C-API. Ideally, all other APIs would then need to be made private. However, I doubt that this is possible before switching to Python 4.0.

Then again, I'm not sure whether that's what you're aiming for...

An optional cross-version ABI would certainly be a good thing.

Limiting the Python C-API would be counterproductive.

During the compilation of applications, the preprocessor macro PyLIMITEDAPI must be defined. Doing so will hide all definitions that are not part of the ABI.

So extensions wanting to use the full Python C-API as documented in the C-API docs will still be able to do this, right ?

Type Objects ------------

The structure of type objects is not available to applications; declaration of "static" type objects is not possible anymore (for applications using this ABI).

Hmm, that's going to create big problems for extensions that want to expose a C-API for their types: Type checks are normally done by pointer comparison using those static type objects.

Functions and function-like Macros ----------------------------------

Function-like macros (in particular, field access macros) remain available to applications, but get replaced by function calls (unless their definition only refers to features of the ABI, such as the various Check macros)

Including Py_INCREF()/Py_DECREF() ?

Excluded Functions ------------------

Functions declared in the following header files are not part of the ABI: - cellobject.h - classobject.h - code.h - frameobject.h - funcobject.h - genobject.h - pyarena.h - pydebug.h - symtable.h - token.h - traceback.h

I don't think that's feasable: you basically remove all introspection functions that way.

This will need a more fine-grained approach.

Linkage -------

On Windows, applications shall link with python3.dll;

You mean: extensions that were compiled with Py_LIMITED_API, right ?

an import library python3.lib will be available. This DLL will redirect all of its API functions through /export linker options to the full interpreter DLL, i.e. python3y.dll.

What if you mix extensions that use the full C-API with ones that restrict themselves to the limited version ?

Would creating a Python object in a full-API extension and free'ing it in a limited-API extension cause problems ?

Implementation Strategy =======================

This PEP will be implemented in a branch, allowing users to check whether their modules conform to the ABI. To simplify this testing, an additional macro PyLIMITEDAPIWITHTYPES will expose the existing type object layout, to let users postpone rewriting all types. When the this branch is merged into the 3.2 code base, this macro will be removed.

Now I'm confused again: this sounds a lot like you do want all extension writers to only use the limited API.

[And in another post]

Something I haven't seen explicitly mentioned as yet (in the PEP or the > python-dev list discussion) are the memory management APIs and the FILE* > APIs which can cause the MSVCRT versioning issues on Windows. > > Those would either need to be excluded from the stable ABI or else > changed to use opaque pointers. Good point. As a separate issue, I would actually like to deprecate, then remove these APIs. I had originally hoped that this would happen for 3.0 already, alas, nobody worked on it. In any case, I have removed them from the ABI now.

How do you expect Python extensions to allocate memory and objects in a platform independent way without those APIs ?

And as an aside: Which API families are you referring to ? PyMem_Malloc, PyObject_Malloc, or PyObject_New ?

Thanks,

Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, May 25 2009)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

2009-06-29: EuroPython 2009, Birmingham, UK 34 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/

More information about the Python-Dev mailing list