[Python-Dev] Weak dict iterators are fragile (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Sun Oct 11 17:50:54 CEST 2009

Hello,

In py3k, the weak dict methods keys(), values() and items() have been changed to return iterators (they returned lists in 2.x). However, it turns out that it makes these methods quite fragile, because a GC collection can occur whenever during iterating, destroy one of the weakref'ed objects, and trigger a resizing of the underlying dict, which in turn raises an exception ("RuntimeError: dictionary changed size during iteration").

This has just triggered (assuming the diagnosis is correct) a hang in test_multiprocessing: http://bugs.python.org/issue7060

I would like to propose some possible solutions against this:

  1. Add the safe methods listkeys(), listitems(), listvalues() which would behave as the keys(), etc. methods from 2.x

  2. Make it so that keys(), items(), values() atomically build a list of items internally, which makes them more costly for large weak dicts, but robust.

What do you think?

Regards

Antoine.

More information about the Python-Dev mailing list