| msg60484 - (view) |
Author: Donovan Baarda (abo) * |
Date: 2004-05-07 05:26 |
| Currently file object's write() method returns nothing. It would be very nice if it returned the number of bytes written. It would also be nice if file objects had a setblocking() method, similar to socket objects. Under various situations, such as communicating with popen'ed processes, write() can block. To avoid this, you must set the file to non-blocking mode, or do something like fork a thread to service the process so that it can "unblock" the write. Currently, setting a file to unblocking mode requires using fnctl on the file objects fileno(). It would be nice if you could directly modify this file behaviour using a setblocking() method. When you do set a file to non-blocking mode, the file object's write() method will raise an exception if not all the data could be written. This is useless, as there is no indication how much data was successfuly written before the blocking happened, so you have no idea how much data remains to be written when the file becomes unblocked. Even using things like select() loops are not reliable, as write() on an unblocked file can block part way through even a small data buffer. The only sure way to avoid a file object's write() from blocking or raising an unhelpful exception is to write a single byte at a time. This means you must use os.write() instead. The os.write() method returns the number of bytes written, which tells how many bytes of data remain to be written next time when the file becomes unblocked. The existing read() method behaves the same as the os.read() method and can be used in non-blocking mode. If the file write() method behaved the same as the os.write() method, returning the number of bytes written instead of raising an exception, then it could also be used in non-blocking mode. |
|
|
| msg82049 - (view) |
Author: Daniel Diniz (ajaksu2) *  |
Date: 2009-02-14 13:54 |
| Implemented in 3.0, 2.x probably won't get this. Will close if nobody opposes. |
|
|
| msg82473 - (view) |
Author: Antoine Pitrou (pitrou) *  |
Date: 2009-02-19 13:53 |
| setblocking() doesn't exist in py3k either, so reopening. I agree it would be useful to set files as non-blocking in a portableway. |
|
|
| msg113427 - (view) |
Author: Giampaolo Rodola' (giampaolo.rodola) * |
Date: 2010-08-09 16:51 |
| Any clue on where to start to do this? What about Windows where fcntl is not available? |
|
|
| msg113431 - (view) |
Author: Tim Golden (tim.golden) * |
Date: 2010-08-09 17:45 |
| There are at least two ways to do non-blocking file IO on Windows: Overlapped I/O I/O Completion ports Don't know what's best here, but happy to see what might be achieved if it was thought worth pursuing. |
|
|
| msg119704 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2010-10-27 12:19 |
| Prototype to test nonblocking file objet: - add getblocking() and setblocking() methods to _io._FileIO and all _pyio classes - fileio_setblocking() is implemented using fcntl(fd, F_SETFL, flags | O_NONBLOCK) (POSIX only?) - BufferedReader.read() uses read1() if the file is blocking Use test_process.py to test it: this script runs a Python interpreter in a subprocess. It uses select() to check if there is data or not. Eg. type '1+1\n' and then 'exit()\n'. Set PYIO_HAVE_BLOCKING constant to False (in test_process.py) to test the script without io_blocking.patch. I'm not sure that select() is required, but it doesn't work without it (read() blocks). |
|
|
| msg178201 - (view) |
Author: Charles-François Natali (neologix) * |
Date: 2012-12-26 11:20 |
| I'm not sure that a setblocking() method to fileobjects would make much sense, since non-blocking IO doesn't work with regular files (only pipes, sockets...). |
|
|
| msg192580 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2013-07-07 18:47 |
| See the PEP 466 which proposes to add a new os.set_blocking() function on UNIX, and blocking parameter to socket constructor. |
|
|
| msg201115 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2013-10-24 11:27 |
| There is a real need of non-blocking operation, but I now think that adding a set_blocking() method is not the right solution. First, as said by Charles-Francois: O_NONBLOCK flag has no effect on regular files. Second, there is no portable way to declare a file as "non blocking": as said by Tim, it is complelty different on Windows (I/O Completion ports). The asyncio module has been merged into Python 3.4: you can now use it to access a file asynchroniously. I don't think that the asyncio is complete for your use case, but if something should done: it is in this module, only in this module, not the API of the io.FileIO class. asyncio can be used to watch pipes of a subprocess asynchroniously. |
|
|
| msg201118 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2013-10-24 12:09 |
| > There is a real need of non-blocking operation, but I now think that > adding a set_blocking() method is not the right solution. First, as > said by Charles-Francois: O_NONBLOCK flag has no effect on regular > files. Second, there is no portable way to declare a file as "non > blocking": as said by Tim, it is complelty different on Windows (I/O > Completion ports). That it's not portable is actually a good reason to add a helper function or method. |
|
|