| msg137260 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2011-05-30 12:38 |
| The following code fails on an assertion error (Python exception for _pyio, C assertion for io): ------------------ with io.BytesIO(b'abcd') as raw: with _pyio.TextIOWrapper(raw, encoding='ascii') as f: f.read(1) f.write('2') f.tell() ------------------ I found this assertion while testing interlaced read-write on TextIOWrapper: ------------------ with io.BytesIO(b'abcd') as raw: with _pyio.TextIOWrapper(raw, encoding='ascii') as f: f.write("1") # read() must call writer.flush() assertEqual(f.read(1), 'b') # write() must rewind the raw stream f.write('2') assertEqual(f.read(), 'd') f.flush() assertEqual(raw.getvalue(), b'1b2d') with io.BytesIO(b'abc') as raw: with _pyio.TextIOWrapper(raw, encoding='ascii') as f: self.assertEqual(f.read(1), b'a') # write() must undo reader readahead f.write(b"2") assertEqual(f.read(1), b'c') f.flush() assertEqual(raw.getvalue(), b'a2c') ------------------ These tests fails on "read, write, read" path: write() breaks TextIOWrapper internal state if a read() occured just before. Note that _pyio.TextIOWrapper.write() contains the following comment... # XXX What if we were just reading? See also the issue #12213 for BufferedRandom and BufferedRWPair. |
|
|
| msg137262 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2011-05-30 13:07 |
| textiowrapper_interlaced_read_write.patch: TextIOWrapper.write() calls self.seek(self.tell()) if it has a decoder or if snapshot is not None. I suppose that we can do better, but at least it does fix this issue. "read(); write(); write()" only calls self.seek(self.tell()) once, at the first write. So I don't think that it changes anything with performances. In which case snapshot can be not different than None, whereas decoder is None? |
|
|
| msg137596 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2011-06-03 22:00 |
| For c stdio files, intermixed reads and writes require a file positioning operation. This is a nuisance and source of program bugs. I do not see any such limitation documented for our io module. So for both reasons, it will be nice to not have the limitation in the code. If I understand, the essence of the patch is to do the file positioning automatically internally when needed. |
|
|
| msg137598 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2011-06-03 22:02 |
| > If I understand, the essence of the patch is to do > the file positioning automatically internally when needed. My patch is just a proposition to fix the issue. I wrote "I suppose that we can do better": self.seek(self.tell()) is more a workaround than a real fix. I don't understand why it does fix this bug :-) |
|
|
| msg137616 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2011-06-04 00:53 |
| Perhaps the stdio requirement was based on an underlying OS (*nix?) requirement, which io has to fulfill even if it does not use stdio. Stdio was, I presume, optimized for speed. In the relatively rare case of mixed read/write, it *should* put the burden on the programmer. Python is a bit different. |
|
|
| msg221439 - (view) |
Author: Mark Lawrence (BreamoreBoy) * |
Date: 2014-06-24 08:54 |
| Does anybody want to follow up on this? #12213 was closed as fixed, #12513 is still open. |
|
|
| msg258636 - (view) |
Author: Martin Panter (martin.panter) * |
Date: 2016-01-20 02:53 |
| Consider codecs that maintain an internal buffer (UTF-7) or other state (ISO-2022). When you call TextIOWrapper.read() and then tell(), the I think the returned number is supposed to hold the _decoder_ state, so you can seek back and read again. But I don’t think the number holds any _encoder_ state, so seek() cannot set up the encoder properly for these more awkward codecs. I don’t think it is practical to fix this problem using the incremental codec API. You would need to construct the encoder’s state from the decoder. There are a couple of bugs marked as duplicates of this. What are the real-world use cases? |
|
|
| msg297126 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-06-28 01:38 |
| Given that nobody complains the last 9 years (since Python 3.0 was released), I'm not sure that it's worth it to fix this corner case. If you consider that I'm wrong, please reopen the issue ;-) |
|
|
| msg330367 - (view) |
Author: Martin Panter (martin.panter) * |
Date: 2018-11-24 00:31 |
| For the record, the more recent bug I mentioned was a complaint from 2015 (one and a half years before Victor’s comment). Even if it is not worth supporting writing after reading, the problem could be documented. |
|
|