[Python-Dev] PEP 442 clarification for type hierarchies (original) (raw)

Stefan Behnel stefan_ml at behnel.de
Sun Aug 4 17:59:57 CEST 2013

Stefan Behnel, 04.08.2013 15:24:

Stefan Behnel, 04.08.2013 09:23:

I'm currently catching up on PEP 442, which managed to fly completely below my radar so far. It's a really helpful change that could end up fixing a major usability problem that Cython was suffering from: user provided deallocation code now has a safe execution environment (well, at least in Py3.4+). That makes Cython a prime candidate for testing this, and I've just started to migrate the implementation.

One thing that I found to be missing from the PEP is inheritance handling. The current implementation doesn't seem to care about base types at all, so it appears to be the responsibility of the type to call its super type finalisation function. Is that really intended? Couldn't the super type call chain be made a part of the protocol? Another bit is the exception handling. According to the documentation, tpfinalize() is supposed to first save the current exception state, then do the cleanup, then call WriteUnraisable() if necessary, then restore the exception state. http://docs.python.org/3.4/c-api/typeobj.html#PyTypeObject.tpfinalize Is there a reason why this is left to the user implementation, rather than doing it generically right in PyObjectCallFinalizer() ? That would also make it more efficient to call through the super type hierarchy, I guess. I don't see a need to repeat this exception state swapping at each level. So, essentially, I'm wondering whether PyObjectCallFinalizer() couldn't just set up the execution environment and then call all finalisers of the type hierarchy in bottom-up order. I continued my implementation and found that calling up the base type hierarchy is essentially the same code as calling up the hierarchy for tpdealloc(), so that was easy to adapt to in Cython and is also more efficient than a generic loop (because it can usually benefit from inlining). So I'm personally ok with leaving the super type calling code to the user side, even though manual implementers may not be entirely happy. I think it should get explicitly documented how subtypes should deal with a tpfinalize() in (one of the) super types. It's not entirely trivial because the tpfinalize slot is not guaranteed to be filled for a super type IIUC, as opposed to tpdealloc. I assume the recursive invariant that PyTypeReady() copies it would still hold, though.

Hmm, it seems to me by now that the only safe way of handling this is to let each tp_dealloc() level in the hierarchy call tp_finalize() through PyObject_CallFinalizerFromDealloc(), instead of calling up the stack in tp_finalize(). Otherwise, it's a bit fragile for arbitrary tp_dealloc() functions in base types and subtypes. However, that appears like a rather cumbersome and inefficient design. It also somewhat counters the advantage of having a finalisation step before deallocation, if the finalisers are only called after (partially) cleaning up the subtypes.

ISTM that this feature hasn't been fully thought out...

Stefan

More information about the Python-Dev mailing list