| msg55993 - (view) |
Author: Alex Burr (ajb) |
Date: 2007-09-18 10:37 |
| If you have made a file nonblocking using fcntl.fcntl, .readline() will discard the start of a line if you get EGAIN. It should attach the partial line to the exception somehow - or at least warn the user. I observe this on 2.3.5, but the same code exists in TRUNK |
|
|
| msg55996 - (view) |
Author: Sean Reifschneider (jafo) *  |
Date: 2007-09-18 15:11 |
| Doesn't the exception count as warning the user? We probably don't want to change readline to return a partial line in this case. An exception could be added for EGAIN that includes the partial line. Another option would be to just document the behavior for readline or fcntl. What is the right behavior for a non-blocking readline? Obviously, it can't return a None and buffer the line. Another option would be to define readline as a blocking operation and enable buffering before starting the readline, and possibly revert it when done. Opinions? |
|
|
| msg55998 - (view) |
Author: Alex Burr (ajb) |
Date: 2007-09-18 15:40 |
| The exception would count as a warning if it wasn't EGAIN. One expects to catch EGAIN and try again. The current situation is unfortunate because it *nearly* works. My scenario is: I'm driving GDB/MI via popen2.Popen3 ( gdbCommand, False,1). It works for most GDB commands, but sometimes GDB returns a huge line and I get EGAIN in the middle. (For my purposes, I've worked round it, by avoiding the command which generates a long line) |
|
|
| msg55999 - (view) |
Author: Sean Reifschneider (jafo) * |
Date: 2007-09-18 15:46 |
| Why are you putting the file in non-blocking mode? Why not just reading in blocking mode? If you want to do other work when a line is not available, you could use select to check to see if there's data ready via a small or 0 timeout. |
|
|
| msg56000 - (view) |
Author: Alex Burr (ajb) |
Date: 2007-09-18 16:25 |
| I don't need to change my code as I have now worked round the problem, but for an example of why someone might want nonblocking mode: It was not actually in order to do any more work, but because I was managing both input and output to gdb in one thread (for simplicity). I therefore needed to collect everything gdb had printed out, but not try to read any more and cause a block because then I could not give GDB its next command, and gdb would not give me any more until I did. I think I did try select, but if I recall correctly, that doesn't work well with .readline() either. I can't remember why, though. |
|
|
| msg56015 - (view) |
Author: Sean Reifschneider (jafo) * |
Date: 2007-09-18 21:24 |
| Arguably, you should be using "select" and "read" (instead of readline) for this operation. That's what I've done in the past when doing something similar. Specifically, I believe I have looped reading into a buffer with read, and using select with a timeout (even a fraction of a second can work) until the timeout triggers. This way I don't block, but I also don't run into problems with the system call returning an error. I avoid using readline in this case because I don't expect it to work well on the stdout of an interactive command. You may also want to look at the pty module. I guess I'd say this should probably go to the python mailing list for further discussion. Can you post a message there about it? |
|
|
| msg56041 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2007-09-19 17:22 |
| readline() goes through C stdio which makes it impossible to get non-blocking I/O right. You should be using raw os.read() calls (until python 3000 which will remove Python's reliance on C stdio). |
|
|