| msg10212 - (view) |
Author: Ralf Juengling (rjuengling) |
Date: 2002-04-08 12:24 |
| A new Python type/class _A is implemented in C. It needs a special allocator function, thus tp_alloc=_A_alloc for this type. There is no need for a special new-method, however, thus tp_new=PyType_GenericNew for this type. The class A inherits from _A and is implemented in Python: class A(_A): pass When an instance of _A is created, its allocator function _A_alloc gets invoked. When an instance of A is created, the _A_alloc does not get invoked. PyType_GenericNew should also invoke all allocator functions of the argument type's superclasses (in reverse mro, I think). |
|
|
| msg10213 - (view) |
Author: Guido van Rossum (gvanrossum) *  |
Date: 2002-04-15 00:09 |
| Logged In: YES user_id=6380 > PyType_GenericNew should also invoke all allocator > functions of the argument type's superclasses (in > reverse mro, I think). No, this doesn't make sense; you can't invoke more than one allocator. (Each allocator would create a whole new object.) The cause of your problem is in type_new(): it always overrides tp_alloc with PyType_GenericAlloc. You'd have to write a new metaclass (inheriting from type) in C to change the tp_alloc (and tp_free) settings. But why do you want this? Why do you want A instances to use the same allocator as _A? This is not how subclassing built-in types works elsewhere; e.g. int uses a special allocator, but subclasses of int don't inherit that. Before I do anything about this I'd like to understand what you are trying to accomplish. |
|
|
| msg10214 - (view) |
Author: Ralf Juengling (rjuengling) |
Date: 2002-04-15 08:06 |
| Logged In: YES user_id=495820 The class A does not know how to allocate the 'base part' of instances of itself (i.e. what makes an instance of class _A). Thus sometime during instantiation of class A objects, the allocater of the base class must be called! This will not happen, however, if the tp_new-slot function merely calls the allocator of the argument type. You pointed out, that e.g. subclasses of 'int' won't inherit int's special allocator. Does this mean, int's allocator will not be invoked on instantiation of the subclass? Then, I don't understand why this works; I can only imagine, that PyType_GenericAlloc is designed to deal with Python built-in types (and probably compostitions of these). |
|
|
| msg10215 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-04-15 13:01 |
| Logged In: YES user_id=6380 Your understanding of instance allocation doesn't match reality, *or* you're expressing yourself poorly. Allocation does not include initialization; allocation simply requests the memory for the object from malloc (or some other memory allocator) and fills it with zeros, except for the reference count, which is set to 1, and the ob_type field, which is set to the type argument to the allocator. The amount of memory allocated for the object is calculated by taking the tp_basicsize of the type object, and adding nitems times the tp_itemsize from the type object. If you want to initialize your object, that should be done in the tp_new slot, not in tp_alloc. Have you tried to read the source code (or go through an allocation in a C debugger) to find out how it really works? |
|
|
| msg10216 - (view) |
Author: Ralf Juengling (rjuengling) |
Date: 2002-04-15 13:46 |
| Logged In: YES user_id=495820 The data structures used internally by class _A cannot by cast into the 'basicsize\itemsize' scheme (i.e. a sparse matrix type). So tp_itemsize actually is set to zero for type _A and its allocator invokes malloc to request memory for the initial data structure (which is going to grow dynamically when a sparse matrix is build up in a Python program). Why would we need the tp_alloc slot at all, if the 'memory demands' of any class (its instances respectively) would fit into the 'basicsize\itemsize'- scheme? I just looked into the source for the 'int' object. The tp_alloc slot is set to zero, allocation is done in 'int_new'. (so it didn't enlighten me...) |
|
|
| msg10217 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-04-15 13:59 |
| Logged In: YES user_id=6380 I'm closing this now, because I don't see the bug in Python. I do see the bug in your approach. You can't dynamically grow the memory allocated for an object! That would move the object in memory and would invalidate all pointers to the object. The right approach is to have a fixed-size object that contains a pointer to separately allocated memory. (Like the list and dict object do.) BTW the tp_alloc slot is needed because some types have a faster allocator, not to improve upon the initial allocation size calculation. |
|
|
| msg10218 - (view) |
Author: Ralf Juengling (rjuengling) |
Date: 2002-04-15 14:32 |
| Logged In: YES user_id=495820 > ...The right approach is to have a fixed-size object > that contains a pointer to separately allocated memory. This is how it works. I admit, I expressed myself poorly here. Do you affirm at the end, that types with dynamic memory requirements cannot be subclassed at the Python level, unless one also implements a special function for the tp_new slot -- to make sure that the base class allocator gets invoked... |
|
|
| msg10219 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-04-15 14:37 |
| Logged In: YES user_id=6380 > Do you affirm... Yes, but I would phrase it differently. Anything that ploinks stuff in the object (even a pointer to additional allocated memory) should be implemented as part of tp_new, which is inherited, rather than tp_alloc. Allocating additional memory is not part of tp_alloc's responsibilities, it's part of tp_new. |
|
|