[Python-Dev] Weak dict iterators are fragile (original) (raw)

Daniel Stutzbach daniel at stutzbachenterprises.com
Sun Oct 11 18:22:41 CEST 2009

On Sun, Oct 11, 2009 at 10:50 AM, Antoine Pitrou <solipsis at pitrou.net>wrote:

In py3k, the weak dict methods keys(), values() and items() have been changed to return iterators (they returned lists in 2.x). However, it turns out that it makes these methods quite fragile, because a GC collection can occur whenever during iterating, destroy one of the weakref'ed objects, and trigger a resizing of the underlying dict, which in turn raises an exception ("RuntimeError: dictionary changed size during iteration").

Ouch!

The iterator from iter is also affected.

  1. Add the safe methods listkeys(), listitems(), listvalues() which would

    behave as the keys(), etc. methods from 2.x

    2. Make it so that keys(), items(), values() atomically build a list of items internally, which makes them more costly for large weak dicts, but robust.

    -1 on 1. +0 on 2.

It'd be nice if we could postpone the resize if there are active iterators, but I don't think there's a clean way to track the iterators.

-- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20091011/d674ae2a/attachment.htm>

More information about the Python-Dev mailing list