[Python-Dev] Threading idea -- exposing a global thread lock (original) (raw)

Raymond Hettinger raymond.hettinger at verizon.net
Tue Mar 14 06:36:19 CET 2006

[Guido]

Oh, no!

Before shooting this one down, consider a simpler incarnation not involving the GIL. The idea is to allow an active thread to temporarily suspend switching for a few steps:

threading.stop_switching()

step1

step2

setp3

theading.resume_switching()

To me, this version is innocuous -- it doesn't violate any existing guarantees (no thread knows when it is going to be switched out or how long it is going to be suspended) and allows the user to easily create an atomic transaction (i.e. swapping two global variables or printing a debug value).

While it could be abused, it has the offsetting virtue of being much simpler than the alternatives.

I disagree that the need is rare. My own use case is that I sometimes add some debugging print statements that need to execute atomically -- it is a PITA because PRINT_ITEM and PRINT_NEWLINE are two different opcodes and are not guaranteed to pair atomically. The current RightWay(tm) is for me to create a separate daemon thread for printing and to send lines to it via the queue module (even that is tricky because you don't want the main thread to exit before a print queued item is completed). I suggest that that is too complex for a simple debugging print statement. It would be great to simply write:

with other_threads_suspended():
    print value

I haven't worked through the IronPython and Jython points of view but think there is probably a way to implement the simpler incarnation.

Also, unlike the gil.aquire() version, there is no blocking of the currently active thread. IOW, the above incarnation won't deadlock (because the main thread never goes into a wait mode).

Raymond

----- Original Message ----- From: "Guido van Rossum" <guido at python.org> To: "Raymond Hettinger" <python at rcn.com> Cc: <python-dev at python.org> Sent: Tuesday, March 14, 2006 12:06 AM Subject: Re: [Python-Dev] Threading idea -- exposing a global thread lock

Oh, no! Please!

I just had to dissuade someone inside Google from the same idea.

IMO it's fatally flawed for several reasons: it doesn't translate reasonably to Jython or IronPython, it's really tricky to implement, and it's an invitation for deadlocks. The danger of this thing in the wrong hands is too big to warrant the (rare) use case that can only be solved elegantly using direct GIL access.

--Guido

On 3/13/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:

A user on comp.lang.python has twisted himself into knots writing multi-threaded code that avoids locks and queues but fails when running code with non-atomic access to a shared resource. While his specific design is somewhat flawed, it does suggest that we could offer an easy way to make a block of code atomic without the complexity of other synchronization tools:

gil.acquire() try: #do some transaction that needs to be atomic finally: gil.release() The idea is to temporarily suspend thread switches (either using the GIL or a global variable in the eval-loop). Think of it as "non-cooperative" multi-threading. While this is a somewhat rough approach, it is dramatically simpler than the alternatives (i.e. wrapping locks around every access to a resource or feeding all resource requests to a separate thread via a Queue). While I haven't tried it yet, I think the implementation is likely to be trivial. FWIW, the new with-statement makes the above fragment even more readable: with atomictransaction(): # do a series of steps without interruption

Raymond

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list