| msg140720 - (view) |
Author: Matt Joiner (anacrolix) |
Date: 2011-07-20 05:24 |
| >>> a = subprocess.Popen(['cat', '/path/to/text.ini'], stdout=subprocess.PIPE, universal_newlines=True) >>> b = configparser.ConfigParser() >>> b.read_file(a.stdout) Traceback (most recent call last): File "", line 1, in File "/hostname/sig/local/lib/python3.2/configparser.py", line 708, in read_file self._read(f, source) File "/hostname/sig/local/lib/python3.2/configparser.py", line 994, in _read for lineno, line in enumerate(fp, start=1): AttributeError: '_io.FileIO' object has no attribute 'read1' Also this fails without universal_readlines, which is not so bad except that the name 'read_file' doesn't exactly indicate this. I found one mildly related bug: http://bugs.python.org/issue11670 |
|
|
| msg140726 - (view) |
Author: R. David Murray (r.david.murray) *  |
Date: 2011-07-20 12:06 |
| This isn't a problem with ConfigParser, it's a problem with the text file object returned by Subprocess. You can reproduce the bug by trying to iterate over the TextIOWrapper returned by subprocess. (It is true, however, that the ConfigParser docs should be specific that it isn't "any file", but "any text file". If you would like to open a separate issue about that, that would be great). |
|
|
| msg140849 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-07-22 00:48 |
| The "universal_newlines" feature is rather poorly named in Python 3.x, since it does much more than that (the resulting files yield and expect unicode strings rather than bytes objects). The problem is that io.TextIOWrapper needs a buffered I/O object, but bufsize is 0 by default in subprocess.Popen(). In the meantime, you can workaround it using a non-zero bufsize in the Popen() constructor. Of course, this has the side-effect of forcing you to call flush() on the stdin stream (if you are using it). |
|
|
| msg140856 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2011-07-22 01:53 |
| So this is a doc bug in subprocess? Explaining this clearly doesn't sound like much fun... |
|
|
| msg140998 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-07-23 17:57 |
| So, as the title indicates, I think we should make TextIOWrapper work with raw IO objects. The reason is so that write() can behave in a totally unbuffered way, which is necessary for Popen to behave appropriately. |
|
|
| msg141000 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-07-23 18:16 |
| Hmm, one problem is that the C TextIOWrapper buffers writes. We would need an additional constructor parameter to prevent that :/ |
|
|
| msg141002 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-07-23 18:33 |
| Another possibility is to provide read1() on RawIO, as a synonym of read(). |
|
|
| msg141003 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-07-23 19:03 |
| Here is a first patch allowing TextIOWrapper to work with raw IO objects, and adding a "write_through" flag. |
|
|
| msg141005 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2011-07-23 19:14 |
| "write_through" is not used in _pyio.py, is it expected? |
|
|
| msg141006 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-07-23 19:17 |
| > "write_through" is not used in _pyio.py, is it expected? Yup, because it is actually always write-through (writes are not cached). The argument is only there so that the signature is the same as the C version. |
|
|
| msg141008 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2011-07-23 19:37 |
| Looks good, then. |
|
|
| msg141009 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-07-23 19:41 |
| This additional patch improves universal_newlines support in subprocess (still broken with the select- and poll-based loops in communicate()). |
|
|
| msg141011 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2011-07-23 19:52 |
| New changeset 9144014028f3 by Antoine Pitrou in branch '3.2': Issue #12591: Allow io.TextIOWrapper to work with raw IO objects (without http://hg.python.org/cpython/rev/9144014028f3 New changeset c3b47cdea0d1 by Antoine Pitrou in branch 'default': Issue #12591: Allow io.TextIOWrapper to work with raw IO objects (without http://hg.python.org/cpython/rev/c3b47cdea0d1 |
|
|
| msg141013 - (view) |
Author: Roundup Robot (python-dev) |
Date: 2011-07-23 20:06 |
| New changeset 5cc536fbd7c1 by Antoine Pitrou in branch '3.2': Issue #12591: Improve support of "universal newlines" in the subprocess http://hg.python.org/cpython/rev/5cc536fbd7c1 New changeset b616396fa170 by Antoine Pitrou in branch 'default': Issue #12591: Improve support of "universal newlines" in the subprocess http://hg.python.org/cpython/rev/b616396fa170 |
|
|
| msg141015 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-07-23 20:12 |
| Ok, this should be fixed now. universal newlines support is still broken with communicate(), I've opened for that. |
|
|