[Python-Dev] PyObject_GC_UnTrack() no longer reliable in 2.7? (original) (raw)

Tim Peters tim.peters at gmail.com
Fri Sep 24 21:14:32 CEST 2010

Looks like 2.7 changes introduced to exempt dicts and tuples from cyclic gc if they obviously can't be in cycles has some unintended consequences. Specifically, if an extension module calls PyObject_GC_UnTrack() on a dict it does not want tracked, Python can start tracking the dict again.

I assume this is unintended because (a) the docs weren't changed to warn about this; and, (b) it's wrong ;-)

There are two main reasons an extension module may have been calling PyObject_GC_UnTrack():

  1. For correctness, if the extension is playing games with reference counts Python isn't expecting.

  2. For speed, if the extension is creating dicts (or tuples) it knows cannot participate in cycles.

This came up when Jim Fulton asked me for advice about assertion failures coming out of cyclic gc in a debug build when running ZODB's tests under 2.7. Turned out to be due to the "#1 reason" above: ZODB hand-rolled its own version of weak references long before Python had them, and has a dict mapping strings ("object IDs") to complex objects where the referenced objects' refcounts intentionally do not account for the references due to this dict.

This had been working fine for at least 8 years, thanks to calling PyObject_GC_UnTrack() on this dict right after it's created.

But under 2.7, new code in Python apparently decides to track this dict again the first time its setitem is called. Cyclic gc then discovers object references due to this dict that aren't accounted for in the referenced objects' refcounts, and dies early on with an assertion failure (which it should do - the refcounts are nuts as far as Python is concerned).

Jim wormed around that for now by calling PyObject_GC_UnTrack() again every time the dict's content changes, but that was relatively easy in this case because the dict is an internal implementation detail all accesses to which are under ZODB's control.

Best if no changes had been needed. "Better than nothing" if the docs are changed to warn that the effect of calling PyObject_GC_UnTrack() may be undone by Python a nanosecond later ;-)

More information about the Python-Dev mailing list