[Python-Dev] PyWeakref_GetObject() borrows its reference from... whom? (original) (raw)

Nathaniel Smith njs at pobox.com
Mon Oct 10 13:50:25 EDT 2016

On Mon, Oct 10, 2016 at 10:03 AM, Paul Moore <p.f.moore at gmail.com> wrote:

On 10 October 2016 at 17:49, MRAB <python at mrabarnett.plus.com> wrote:

If you lookup something in a dict, it'll be a borrowed reference.

If the dict is globals() and there's no GIL, another thread could delete the item while your code had the borrowed reference. It looks like there might be a lot that will need to changed post gilectomy! It seems to me that the whole concept of a borrowed reference may be risky in a post-GIL world. There may be occasional cases where it's possible to prove that borrowing is safe, but I suspect they'll be pretty rare.

I don't think it's that bad...

In a post-GIL world, the problem cases we're talking about like deleting items from containers all require some sort of locking, even before we start thinking about borrowed references. If two threads start mucking around resizing the internals of a list or dict at the same time, then you are unlikely to go to space today. IIRC to handle this gilectomy adds per-object mutexes that you have to hold whenever you're mucking around with that object's internals.

If we say that borrowing reference from a dict is one of the things that counts as mucking about with that dict, and thus requires you to hold the dict lock for as long as you hold the borrowed reference, then all should be well.

I assume Larry is way ahead of us on this, which is why he's suddenly confusing us all by emphasizing that when you borrow a reference you should know who you're borrowing it from, so you know which lock to hold.

-n

-- Nathaniel J. Smith -- https://vorpus.org

More information about the Python-Dev mailing list