[Python-Dev] RE: Evil Trashcan and GC interaction (original) (raw)

Tim Peters tim.one@comcast.net
Thu, 28 Mar 2002 17:08:43 -0500

[Neil Schemenauer]

That's the same fix Andrew and I had in mind. My concern is that this is probably not the only bug of this type. The trashcan mechanism changes the ordering of object deallocation. What are the chances of other bugs like this lurking somewhere?

I'll first expand on what "bug of this type" means: Any use of any C API gimmick G is a bug (discovered or not) if (sufficient but not necessary):

  1. G can (directly or indirectly) call back into Python code;

and

2A. The code using G leaves a reachable object in an inconsistent state; and/or

2B. The code using G assumes (and usually implicitly) that mutable portions of a reachable object won't change across its use of G.

I've lost track of how many bugs of this sort we've fixed over the years. In an odd sense, it's a consequence of the GIL: a "free threaded" interpreter would get burned by "this kind of thing" all the time. The GIL restricts the possibilities for this kind of damage to specific call sites, but exactly which sites changes across releases. For a dramatic example, before cyclic GC was added, PyTuple_New was as harmless as water (too shallow to drown in, and not moving at lethal speed ). Now it can lead to gc and destructor invocation, thread switches, and move list guts around in memory. As PyDict_Items learned the hard way, allocating a 2-tuple for a key/value pair can even resize the dict it's looking at.

I'm certain there are more vulnerabilities in the core, but I bet we've fixed way more than half . The SETLOCAL bug is a #2A bug that's been there forever, yet even know that we know about it, I don't believe it's been the cause of any mystery-failure I ever heard of.

So rather than asking what the odds are of other bugs like this (my guess is 99+% there's at least one more), I wonder what the odds are that anyone will write a program that cares. The odds of someone writing a program like Zope are too small to measure .