Issue 1444: utf_8_sig streamreader bug, patch, and test (original) (raw)

Created on 2007-11-15 08:06 by jgsack, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (5)
msg57520 - (view)	Author: James G. sack (jim) (jgsack)	Date: 2007-11-15 08:06
The streamreader in utf_8_sig.py fails when asked to read a specified bytelength of data that ends up in the middle of a multibyte utf8 code. I will attached a atandalone unittest (which does work from autotest, but doesn't use test_support), test_utf8sig_stream.py. I will attach a patch (applied to the trunk 2.6 version), u8sig26.diff. Regards, ..jim
msg57523 - (view)	Author: James G. sack (jim) (jgsack)	Date: 2007-11-15 09:12
Oops, it looks like my patch may have broken test_partial in test_codecs. I will try to figure out what the test_partial does in the next day or so, unless someone else can add some insignt in the meantime. .jim
msg57524 - (view)	Author: James G. sack (jim) (jgsack)	Date: 2007-11-15 09:48
One additional clue: test_codecs succeeds in verbose mode but fails in non- verbose mode (autotest "verbosity") .. I think. My eyes are getting blurry. More tomorrow, I guess. ..j
msg57582 - (view)	Author: James G. sack (jim) (jgsack)	Date: 2007-11-16 04:50
I found the errror in my previous patch. It lacked a self.decode=.. line in the StreamReader.decode elif branch. I attach a replacement patch diff-u.py26_utf8sig (apply to the 2.6 version of utf_8_sig.py. (If allowed, I will next remove the incorrect patch.) This one passes test_codecs.py as well as my previously attached test module. The resulting utf_8_sig.py may benefit from further refctoring, but I didn't want to do more than necessary to fix the immediate bug. Regards, ..jim
msg57633 - (view)	Author: Walter Dörwald (doerwalter) *	Date: 2007-11-19 12:50
Checked in your change and the test as r59049 (trunk) and r59050 (2.5). Thanks for the patch.