[Python-Dev] Thread-safe file objects, the return (original) (raw)

Gregory P. Smith greg at krypto.org
Sun Apr 6 08:29:29 CEST 2008

I've reviewed the patch on http://bugs.python.org/issue815646 and have uploaded my modified version (mostly test improvements and some formatting to keep C code under 80 columns with proper 8 space tabs). I would have committed it already but I have a sneaking suspicion that its unit test will barf on windows since it could depend on some posix-like file system semantics.

Could someone with a windows build environment please test it as asked in the issue and report back in the tracker?

thanks! -gps

On Wed, Apr 2, 2008 at 11:47 AM, Guido van Rossum <guido at python.org> wrote:

On Wed, Apr 2, 2008 at 3:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote: > Guido van Rossum <guido python.org> writes: > > Your solution (a counter) seems fine except I think perhaps the > > close() call should not raise IOError -- instead, it should set a flag > > so that the thread that makes the counter go to zero can close the > > thread (after all the file got closed while it was being used). > > I agree with Gregory: we should be explicit about what happens. I wonder > what we would gain from that approach - apart from encouraging dangerous > coding practices :) > It also depends how far we want to go: I am merely proposing to fix the > crashes, do we want to provide a "smarter" close() variation that does what > you suggest for people that want (or need) to take the risk?

I also agree with Gregory now -- at least on the issue of fclose(). I think that for other (non-closing) operations we should be fine, given the Posix requirement that streams have an internal lock. While multiple threads reading from a file sounds insane, multiple files writing to a file is pretty common (think of stdout or stderr) and should be supported. > > There are of course other concurrency issues besides close -- what if > > two threads both try to do I/O on the file? What will the C stdio > > library do in that case? Are stdio files thread-safe at the C level? > > According to the glibc documentation, at > http://www.gnu.org/software/libc/manual/htmlnode/Streams-and-Threads.html: > > « The POSIX standard requires that by default the stream operations are > atomic. I.e., issuing two stream operations for the same stream in two > threads at the same time will cause the operations to be executed as if > they were issued sequentially. The buffer operations performed while > reading or writing are protected from other uses of the same stream. To do > this each stream has an internal lock object which has to be (implicitly) > acquired before any work can be done. » > > So according to POSIX rules it should be perfectly safe. Cool. > In any case, someone would have to try my patch under Windows and OS X and > see if testfile.py passes without crashing. I know Windows has internal locks on stdio. I trust that OSX, being a BSD descendant, is posix compliant. So I'm not worried about these. +1 on your patch, assuming some other developer reviews it and submits it. -- --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/> )

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080405/0427aa16/attachment-0001.htm

More information about the Python-Dev mailing list