Flawed assumptions about tp_dictoffset in inheritance. · Issue #95589 · python/cpython (original) (raw)

In Python, the __dict__ and __weakref__ slots are treated specially (slots meaning __slots__, not tp_slots)
They are automatically insert by the VM when creating a class.

class C(list): pass

C().dict {}

In order to support inheritance, specifically multiple inheritance, the VM can lay out subclasses in ways that differ from the superclass.
This is OK, provided __dict__ and __weakref__ are only accessed though the tp_dictoffset and tp_weaklistoffset offsets.
But, if either field is accessed directly, then we access invalid memory and 💥

test.py:

from _testcapi import HeapCTypeWithDict class I3(HeapCTypeWithDict, list): pass

i = I3() i.append(0) print(i.dictobj)

$ python3.10 ~/test/test.py Segmentation fault (core dumped)

We have (accidentally) fixed this for __dict__ in 3.11, although at the expense breaking backwards compatibility for some C extensions. However, the problem still remains for __weakref__.

Backwards incompatibility

from _testcapi import HeapCTypeWithDict class I3(HeapCTypeWithDict, list): pass print("OK")

$ python3.10 test.py OK $ python3.12 test.py Traceback (most recent call last): File "test.py", line 3, in class I3(HeapCTypeWithDict, list): pass TypeError: multiple bases have instance lay-out conflict