[Python-Dev] Python memory model (low level) (original) (raw)

Tim Peters tim.peters at gmail.com
Fri Jun 30 21:14:24 CEST 2006

[Nick Maclaren]

I Have been thinking about software floating point, and there are some aspects of Python and decimal that puzzle me. Basically, they are things that are wanted for this sort of thing and seem to be done in very contorted ways, so I may have missed something.

Firstly, can Python C code assume no COMPACTING garbage collector, or should it allow for things shifting under its feet?

CPython never relocates objects. The address of a Python object is fixed from birth to death.

Secondly, is there any documentation on the constraints and necessary ritual when allocating chunks of raw data and/or types of variable size? Decimal avoids the latter.

Note that because CPython never relocates objects, it's impossible to allocate "in one gulp" an object of a mutable type with a dynamically size-varying member. So, e.g., list objects have a fixed-size object header (which never moves), while the list "guts" are separately allocated; OTOH, tuples are immutable, so can (& do) allocate all the space they need in one gulp.

See the Python/C API Reference Manual, esp. "Allocating Objects on the Heap", "Supporting Cyclic Garbage Collection", and "Memory Management".

Thirdly, I can't find an efficient way for object-mangling code to access class data and/or have some raw data attached to a class (as distinct from an instance).

Don't know what "raw data" might mean here. Any Python object can be bound to any attribute of a class. In Python, e.g.,

class MyClass:

 mydata = ['xyz', 12]

 def method(self):
     MyClass.mydata.append(-1)
     # or, more inheritance-friendly
     self.__class__.mydata.append(-1)

This is substantially more long-winded in C.

Fourthly, can I assume that no instance of a class will remain active AFTER the class disappears? This would mean that it could use a pointer to class-level raw data.

It would be an error to free the memory for a class if the class is reachable. So long as an instance I of a class C is reachable, C is also reachable (at least via I.class) so C must not disappear. C disappears only if it becomes unreachable, and one of the garbage collection mechanisms then gets around to freeing its memory. All gc mechanisms in CPython rely on accurate reference counts.

I can explain why all of those are the 'right' way to approach the problem, at an abstract level, but it is quite possible that Python does not support the abstract model of class implementation that I am thinking of.

Since we don't know what you're thinking of, you're on your own there ;-)

More information about the Python-Dev mailing list