[Python-Dev] PyObject_GC_UnTrack() no longer reliable in 2.7? (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Fri Sep 24 21:36:00 CEST 2010

On Fri, 24 Sep 2010 15:14:32 -0400 Tim Peters <tim.peters at gmail.com> wrote:

Looks like 2.7 changes introduced to exempt dicts and tuples from cyclic gc if they obviously can't be in cycles has some unintended consequences. Specifically, if an extension module calls PyObjectGCUnTrack() on a dict it does not want tracked, Python can start tracking the dict again.

I assume this is unintended because (a) the docs weren't changed to warn about this; and, (b) it's wrong ;-)

It was indeed unintended. I didn't know people were using PyObject_GC_(Un)Track in other places than constructors and destructors.

There are two main reasons an extension module may have been calling PyObjectGCUnTrack():

1. For correctness, if the extension is playing games with reference counts Python isn't expecting.

Yikes :)

2. For speed, if the extension is creating dicts (or tuples) it knows cannot participate in cycles.

The optimization is now automated in the simple cases (as you've found out!).

This came up when Jim Fulton asked me for advice about assertion failures coming out of cyclic gc in a debug build when running ZODB's tests under 2.7. Turned out to be due to the "#1 reason" above: ZODB hand-rolled its own version of weak references long before Python had them, and has a dict mapping strings ("object IDs") to complex objects where the referenced objects' refcounts intentionally do not account for the references due to this dict.

Perhaps ZODB should switch to standard weak references these days? (as a bonus, chances are it will be faster)

Best if no changes had been needed. "Better than nothing" if the docs are changed to warn that the effect of calling PyObjectGCUnTrack() may be undone by Python a nanosecond later ;-)

A doc addition will be enough, hopefully.

Regards

Antoine.

More information about the Python-Dev mailing list