[Python-Dev] PyWeakref_GetObject() borrows its reference from... whom? (original) (raw)

MRAB python at mrabarnett.plus.com
Mon Oct 10 16:27:38 EDT 2016

On 2016-10-10 20:36, Chris Angelico wrote:

On Tue, Oct 11, 2016 at 5:24 AM, Random832 <random832 at fastmail.com> wrote:

On Mon, Oct 10, 2016, at 14:04, MRAB wrote:

Instead of locking the object, could we keep the GIL, but have it normally released?

A thread could then still call a function such as PyWeakrefGetObject() that returns a borrowed reference, but only if it's holding the GIL. It would be able to INCREF the reference before releasing the GIL again. So, what stops the other thread which never asks for the GIL from blowing away the reference? Or is this a special kind of lock that you can "assert isn't locked" without locking it for yourself, and INCREF/DECREF does so? "assert isn't locked" is pretty cheap (race conditions wouldn't be applicable here, as you would have to completely obtain the GIL before even attempting a dangerous operation), but what would INCREF/DECREF do if the GIL is locked by another thread? Hmm. Here's a naughty, and maybe dangerous, theory. Obtain a "memory deallocation lock". While it is held (by any thread - it's a guard, more than a lock), PyDECREF will not actually deallocate memory - objects can fall to zero references without being wiped. Once the lock/guard is freed/cleared, anything that had fallen to zero is now deallocated. This probably would mean stuffing them onto a list of "doomed objects", and upon release of the guard, any doomed objects that still have no refs would get deallocated. That would, by definition, make all borrowed refs legal; you can either use them and finish, or INCREF them before releasing the deallocation lock, and either way, you have a guarantee that (a) you're not messing with junk memory, and (b) it really is the object you think it is. There should be no risk of race conditions, because you would completely claim the deallocation lock before calling something that gives you a borrowed ref, meaning that the whole time the ref is borrowed, you must have that lock. But I'm guessing other people have thought far more about this than I have. I have to admit that the GIL thing was just a vague feeling I had, and I hadn't thought too deeply about it, but I think that you've crystallised it. :-)

More information about the Python-Dev mailing list