[Python-Dev] PyWeakref_GetObject() borrows its reference from... whom? (original) (raw)

MRAB python at mrabarnett.plus.com
Mon Oct 10 14:04:30 EDT 2016

On 2016-10-10 18:50, Nathaniel Smith wrote:

On Mon, Oct 10, 2016 at 10:03 AM, Paul Moore <p.f.moore at gmail.com> wrote: > On 10 October 2016 at 17:49, MRAB <python at mrabarnett.plus.com> wrote: >> If you lookup something in a dict, it'll be a borrowed reference. >> >> If the dict is globals() and there's no GIL, another thread could delete the >> item while your code had the borrowed reference. >> >> It looks like there might be a lot that will need to changed post gilectomy! > > It seems to me that the whole concept of a borrowed reference may be > risky in a post-GIL world. There may be occasional cases where it's > possible to prove that borrowing is safe, but I suspect they'll be > pretty rare.

I don't think it's that bad... In a post-GIL world, the problem cases we're talking about like deleting items from containers all require some sort of locking, even before we start thinking about borrowed references. If two threads start mucking around resizing the internals of a list or dict at the same time, then you are unlikely to go to space today. IIRC to handle this gilectomy adds per-object mutexes that you have to hold whenever you're mucking around with that object's internals. If we say that borrowing reference from a dict is one of the things that counts as mucking about with that dict, and thus requires you to hold the dict lock for as long as you hold the borrowed reference, then all should be well. I assume Larry is way ahead of us on this, which is why he's suddenly confusing us all by emphasizing that when you borrow a reference you should know who you're borrowing it from, so you know which lock to hold. Instead of locking the object, could we keep the GIL, but have it normally released?

A thread could then still call a function such as PyWeakref_GetObject() that returns a borrowed reference, but only if it's holding the GIL. It would be able to INCREF the reference before releasing the GIL again.

There could still be a new function that returned a strong reference.

More information about the Python-Dev mailing list