[Python-Dev] Thread-safe file objects, the return (original) (raw)

Guido van Rossum guido at python.org
Wed Apr 2 20:47:02 CEST 2008

On Wed, Apr 2, 2008 at 3:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

Guido van Rossum <guido python.org> writes: > Your solution (a counter) seems fine except I think perhaps the > close() call should not raise IOError -- instead, it should set a flag > so that the thread that makes the counter go to zero can close the > thread (after all the file got closed while it was being used).

I agree with Gregory: we should be explicit about what happens. I wonder what we would gain from that approach - apart from encouraging dangerous coding practices :) It also depends how far we want to go: I am merely proposing to fix the crashes, do we want to provide a "smarter" close() variation that does what you suggest for people that want (or need) to take the risk?

I also agree with Gregory now -- at least on the issue of fclose().

I think that for other (non-closing) operations we should be fine, given the Posix requirement that streams have an internal lock. While multiple threads reading from a file sounds insane, multiple files writing to a file is pretty common (think of stdout or stderr) and should be supported.

> There are of course other concurrency issues besides close -- what if > two threads both try to do I/O on the file? What will the C stdio > library do in that case? Are stdio files thread-safe at the C level?

According to the glibc documentation, at http://www.gnu.org/software/libc/manual/htmlnode/Streams-and-Threads.html : « The POSIX standard requires that by default the stream operations are atomic. I.e., issuing two stream operations for the same stream in two threads at the same time will cause the operations to be executed as if they were issued sequentially. The buffer operations performed while reading or writing are protected from other uses of the same stream. To do this each stream has an internal lock object which has to be (implicitly) acquired before any work can be done. » So according to POSIX rules it should be perfectly safe.

Cool.

In any case, someone would have to try my patch under Windows and OS X and see if testfile.py passes without crashing.

I know Windows has internal locks on stdio. I trust that OSX, being a BSD descendant, is posix compliant. So I'm not worried about these.

+1 on your patch, assuming some other developer reviews it and submits it.

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list