[Python-Dev] Is there any remaining reason why weakref callbacks shouldn't be able to access the referenced object? (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Sat Oct 22 06:01:49 EDT 2016

On 22 October 2016 at 16:05, Nathaniel Smith <njs at pobox.com> wrote:

On Fri, Oct 21, 2016 at 8:32 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: But PEP 442 already broke all that :-). Now weakref callbacks can happen before del, and they can happen on objects that are about to be resurrected.

Right, but the resurrection can still only happen in del, so the interpreter doesn't need to deal with the case where it happens in a weakref callback instead - that's where the freedom to do the callbacks and the del in either order comes from.

There remains one obscure corner case where multiple resurrection is possible, because the resurrection-prevention flag doesn't exist on non-GC objects, so you'd still be able to take new weakrefs to those. But in that case del can already do multiple resurrections, and some fellow named Nick Coghlan seemed to think that was okay back in 2013 [1], so probably it's not too bad ;-).

[1] https://mail.python.org/pipermail/python-dev/2013-June/126850.html

Right, that still doesn't bother me.

Changing that to support resurrecting the object so it can be passed into the callback without the callback itself holding a strong reference means losing the main "reasoning about software" benefit that weakref callbacks offer: they currently can't resurrect the object they relate to (since they never receive a strong reference to it), so it nominally doesn't matter if the interpreter calls them before or after that object has been entirely cleaned up. I guess I'm missing the importance of this -- does the interpreter gain some particular benefit from having flexibility about when to fire weakref callbacks? Obviously it has to pick one in practice.

Sorry, my attempted clarification of one practical implication made it look like I was defining the phrase I had in quotes. However, the "reasoning about software" benefit I see is "If you don't define del, you don't need to worry about object resurrection, as it's categorically impossible when only using weakref callbacks". Interpreter implementors are just one set of beneficiaries of that simplification - everyone writing weakref callbacks qualifies as well.

However, if you're happy defining del methods, then PEP 442 means you can already inject lazy cyclic cleanup that supports resurrection:

>>> class Target:
...     pass
...
>>> class Resurrector:
...     def __init__(self, target):
...         _self_ref = "_resurrector_{:d}".format(id(self))
...         self.target = target
...         setattr(target, _self_ref, self)
...     def __del__(self):
...         globals()["resurrected"] = self.target
...
>>> obj = Target()
>>> Resurrector(obj)
<__main__.Resurrector object at 0x7f42f8ae34e0>
>>> del obj
>>> resurrected
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'resurrected' is not defined
>>> import gc
>>> gc.collect(); gc.collect(); gc.collect()
6
4
0
>>> resurrected
<__main__.Target object at 0x7f42f8ae3438>

Given that, I don't see a lot of benefit in making weakref callbacks harder to reason about when del + attribute injection already makes this possible.

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list