[Python-Dev] What is the precise problem? [was: Reference cycles in Exception.traceback] (original) (raw)

Jim J. Jewett jimjjewett at gmail.com
Tue Mar 11 00:08:52 CET 2014

On Mon Mar 10 18:56:17 CET 2014 (and earlier quotes), Maciej Fijalkowski wrote:

Maciej:>> You should not rely on __del__s being called timely one way or Maciej:>> another. Why would you require this for the program to work Maciej:>> correctly in the particular example of traceback?

To the extent that I understand, he isn't requiring it for correctness; he is instead saying that without timely del, the Quality of Implementation suffers.

I suspect there are aspects of tulip (or event processing in general) that make it more common for the frame graph to be painfully cyclic, so that live frames keep dead ones from being collected.

It may also be more common to have multiple del methods in the same cycle, if cycles are created by a framework. So the problems aren't new, but they may have become considerably more painful.

Victor:> For asyncio, it's very useful to see unhandled exceptions as early as Victor:> possible. Otherwise, your program is blocked and you don't know why.

...

Maciej:>> twisted goes around it by attaching errback by hand. Would that work Maciej:>> for tulip?

Maciej:>> deferred.addErrback(callback_that_writes_to_log)

What do you mean "by hand"? Does the framework automatically add a "log the exception" errback to every task, or every task that doesn't have its own errback of some sort? Or do you mean that users should do so by hand, but that it is a well-known recipe?

Maciej:>> I'm very skeptical about changing details of traceback and Maciej:>> frames, just in order to make refcounting work (since it would Maciej:>> create something that would not work on pypy for example).

How about just loosening some constraints on exceptions, in order to permit more efficient operation, but in a way that may be particularly useful to a refcounting scheme? Can I assume that you don't object to frame.clear()? How about a hypothethetical traceback.pack() that made it easier to reclaim memory held by frame/traceback cycles? If standard traceback printing were the only likely future use, each frame/traceback pair could be replaced by 4 pointers, and allocating space/copying those 4 would be the only work that wasn't already needed for the eventual deallocation.

Today, the setters for cause, context, and __traceback do typechecks to ensure that those properties are (None or) the expected type; traceback doesn't even allow subclasses. The constructors for frame and traceback are similarly strict.

What would be the harm in allowing arbitrary objects, let alone a few specific alternative implementations?

-jJ

--

If there are still threading problems with my replies, please email me with details, so that I can try to resolve them. -jJ

More information about the Python-Dev mailing list