[Python-Dev] Cycle collection enhancement idea (original) (raw)

Eyal Lotem eyal.lotem at gmail.com
Sun Jun 29 20:23:34 CEST 2008

On Sun, Jun 29, 2008 at 9:00 PM, "Martin v. Löwis" <martin at v.loewis.de> wrote:

As I explained above, it is part of a cycle: """including the class objects themselves: class->dict->function->funcglobals""". Ah, right. I must have missed that explanation.

I know. I assumed Python does not rely on cyclic garbage collectioin for shutdown, because it wouldn't work, as all globals that have any instance method will be part of a cycle, and any of them which have a del will not be collected. No. The mechanism for cleaning modules at shutdown time predates cyclic GC, and was not removed because "it wouldn't work". This specific issue certainly contributes to the fact that it doesn't work, but there might be other problems as well (such as extension modules holding onto objects participating in cycles). I mentioned this workaround. What I propose is not a workaround but a solution. You wouldn't need to clean up module globals ad-hoc'ishly, because the cyclic collection would collect your object, even with its del. I don't think it solves the problem. You seem to be assuming that any such cycle will contain only a single global object with an del. However, as modules refer to each other, I very much doubt that is the case. Please read my entire mail before replying to it. Thanks! I really, really tried. I read it three times before replying. However, I found it really, really difficult to follow your writing, as it was mixing problem statement and solution, so that I couldn't tell what paragraph was about what. English is not my native language, complicating communication further. Please accept my apologies.

I apologize for my tone as well, it just seemed frustrating that all the replies seemed to completely ignore the core point I was trying to make and claim that the problem I was trying to solve was not a problem at all, but a behavior I was unaware of...

Mixing another quote from another mail:

> That would be no worse than what happens now - but its still not > perfect (del ordering issues). Also, you would need to temporarily > revive the cycles as mentioned above (to avoid accessibility of > partially destructed objects).

I don't quite understand what you mean by "revive cycles". There is not need to revive any object in a cycle, as all objects are still alive (or else they wouldn't be garbage). By "revive cycles", I mean make sure that they are referenced by an independent referrer (one that won't go away as part of the del calling process). This is similar to how the tp_dealloc code increases the refcount (actually sets it to 1, because it was certainly 0 when entering the destructor) before calling the del slot. Without reviving the object before calling its del in the destructor, and without reviving the objects of a cycle before calling its del's, the del Pythonic code may be exposed to "dead objects" (refcount==0).

Consider the cycle: a.x = b b.x = a

Lets suppose the a object has a del. Lets assume each object in the cycle has a refcount of 1 (and the cycle should die). Now lets say this is a's del code: def del(self): self.x = None

Running it will set 'b's refcount to 0 and call its destructor, which will set 'a's refcount to 0 and also call its destructor. But its del is currently running - so "self" must not have a refcount of 0. If you only incref on 'a' before calling del, then you are probably alright, as long as there is only one del.

Can you please elaborate? What would such del ordering issues be?

class A(object): def del(self): print self.x.attribute

class B(object): def del(self): print "B is going down!" del self.attribute

a = A() b = B() a.x = b b.attribute = 1

If you call b's del first then a's del will fail. If you call a's del first, then all is well. Ofcourse you can create true cyclic dependencies that no order will work, and its pretty clear there is no way to deduce the right order anyway. This is what I mean by "ordering issues".

There could be a global barricade for calling del: you first call _all del_s of existing objects, then set the barricade, and then start breaking cycles. This could even be done with the current approach to module clearing.

Note that the del's themselves may be breaking cycles and refcounts will go to 0 - unless you temporarily revive (incref) the entire cycle first.

I still don't understand what "revive the cycle" means. You will need to incref the object for which you call del, that's all.

Unless there are multiple del's in the cycle.

Regards, Martin

More information about the Python-Dev mailing list