PEP 683 (Immortal Objects): Implement Py_INCREF() as function call in limited C API 3.12 · Issue #105387 · python/cpython (original) (raw)
The implementation of Py_INCREF() and Py_DECREF() functions changed in Python 3.12 to implement PEP 683 (Immortal Objects). Stable ABI sections of PEP 683 says:
The implementation approach described in this PEP is compatible with extensions compiled to the stable ABI (with the exception of Accidental Immortality and Accidental De-Immortalizing). Due to the nature of the stable ABI, unfortunately, such extensions use versions of Py_INCREF(), etc. that directly modify the object’s ob_refcnt field. This will invalidate all the performance benefits of immortal objects.
I propose to change the limited C API implementation to use an opaque function call in the limited C API 3.12 and newer. It hides tricky implementation details: see issue #105059 and PR #105275 for a concrete example. It will also ease the hypothetical implementation of PEP 703 "NO GIL".
Converting Py_INCREF() and Py_DECREF() static inline functions to opaque function calls has an impact on performance. IMO the change is worth it. I didn't measure the impact on performance.
In Python 3.10 and 3.11, I modified the Python C API to abstract access to PyObject and PyVarObject with function calls, rather than accessing directly structure members: see issue #83754. Currently, it's not enforced, it's still possible to access directly structure members. This step is another step towards fully opaque PyObject and PyVarObject structure (remove their members from the public API).