Issue 20435: Discrepancy between io.StringIO and _pyio.StringIO with univeral newlines (original) (raw)

Created on 2014-01-29 09:16 by serhiy.storchaka, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
stringio_getvalue.patch pitrou,2014-02-02 21:39
stringio_newline.patch serhiy.storchaka,2014-02-02 22:31 review
Messages (11)
msg209625 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-29 09:16
>>> import io, _pyio >>> io.StringIO('a\nb\r\nc\rd', newline=None) <_io.StringIO object at 0xb707c734> >>> io.StringIO('a\nb\r\nc\rd', newline=None).getvalue() 'a\nb\nc\nd' >>> _pyio.StringIO('a\nb\r\nc\rd', newline=None).getvalue() 'a\nb\r\nc\rd' >>> s = io.StringIO(newline=None); s.write('a\nb\r\nc\rd'); s.getvalue() 8 'a\nb\nc\nd' >>> s = _pyio.StringIO(newline=None); s.write('a\nb\r\nc\rd'); s.getvalue() 8 'a\nb\r\nc\rd'
msg210019 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-02 21:39
It's a bug in _pyio.StringIO.getvalue(). Attaching patch.
msg210028 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-02 22:02
I rather think that it's a bug in _io.StringIO.getvalue().
msg210031 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-02 22:06
With the patch, getvalue() is consistent with read() and other methods. Without the patch, _pyio.StringIO.getvalue() returns a different value from _pyio.StringIO.read() *and* from _io.StringIO.getvalue(). Changing _pyio.StringIO.getvalue() is the path of least resistance here.
msg210033 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-02 22:09
In other words, the bug is that _pyio.StringIO.getvalue() doesn't do any newline conversion; the patch fixes that.
msg210041 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-02 22:31
But how then other values of newline except '\n' can be useful? The problem is that newline converting is applied twice, in write() and in read(). When constructor uses write() and getvalue() returns same value as read(), we have no chance to get newlines encoded or decoded only once. Current results for newline != '\n' looks meanless to me. Here is my half-baked patch. It fixes only _pyio.StringIO, but _io.StringIO are still partially broken. The patch also contains new tests.
msg210043 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-02 22:36
> Current results for newline != '\n' looks meanless to me. They don't look meaningless to me, e.g.: >>> io.StringIO("a\r\nc\rd", newline=None).getvalue() 'a\nc\nd' >>> sio = io.StringIO(newline=None); sio.write("a\r\nc\rd"); sio.getvalue() 6 'a\nc\nd' There may be other cases where they make less sense, but that's a separate issue.
msg210044 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-02-02 22:40
New changeset 99168e7d4a3d by Antoine Pitrou in branch '3.3': Issue #20435: Fix _pyio.StringIO.getvalue() to take into account newline translation settings. http://hg.python.org/cpython/rev/99168e7d4a3d New changeset aadcc71a4967 by Antoine Pitrou in branch 'default': Issue #20435: Fix _pyio.StringIO.getvalue() to take into account newline translation settings. http://hg.python.org/cpython/rev/aadcc71a4967
msg210045 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-02-02 22:42
New changeset 3e61d8e06ef7 by Antoine Pitrou in branch '2.7': Issue #20435: Fix _pyio.StringIO.getvalue() to take into account newline translation settings. http://hg.python.org/cpython/rev/3e61d8e06ef7
msg210047 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-02 22:42
I've fixed the reported bug. If other problems need fixing, better open a new issue :-)
msg210079 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-03 06:39
Thank you for fixing this bug. Maybe I reported it too hastily.
History
Date User Action Args
2022-04-11 14:57:57 admin set github: 64634
2014-02-03 06:39:04 serhiy.storchaka set messages: +
2014-02-02 22:42:50 pitrou set status: open -> closedresolution: fixedmessages: + stage: resolved
2014-02-02 22:42:04 python-dev set messages: +
2014-02-02 22:40:56 python-dev set nosy: + python-devmessages: +
2014-02-02 22:36:54 pitrou set messages: +
2014-02-02 22:31:49 serhiy.storchaka set files: + stringio_newline.patchmessages: +
2014-02-02 22:09:07 pitrou set messages: +
2014-02-02 22:06:56 pitrou set messages: +
2014-02-02 22:02:26 serhiy.storchaka set messages: +
2014-02-02 21:39:07 pitrou set files: + stringio_getvalue.patchkeywords: + patchmessages: +
2014-01-29 09:16:06 serhiy.storchaka create