Message 361513 - Python tracker (original) (raw)

Today, CPython is leaking too many implementation through its public C API. We cannot easily change the "default" C API, but we can enhance the "limited" C API (when Py_LIMITED_API macro is defined). Example of leaking implementation details: memory allocator, garbage collector, structure layouts, etc.

Making PyObject an opaque structure would allow in the long term of modify structures to implement more efficient types (ex: list specialized for small integers), and it can prepare CPython to experiment tagged pointers.

Longer rationale:

• https://pythoncapi.readthedocs.io/
• https://pythoncapi.readthedocs.io/bad_api.html
• https://pythoncapi.readthedocs.io/optimization_ideas.html

I propose to incremental evolve the existing limited C API towards opaque PyObject, by trying to reduce the risk of breakage.

We may test changes on PyQt which uses the limited C API.

Another idea would be to convert some C extensions of the standard library to the limited C API. It would ensure that the limited C API contains enough functions to be useful, but would also notify us directly if the API is broken.