Issue 28758: UnicodeDecodeError: 'gbk' codec can't decode byte 0xab in position 74: illegal multibyte sequence (original) (raw)

Created on 2016-11-21 05:17 by dontbugme, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (5)
msg281322 - (view) Author: (dontbugme) Date: 2016-11-21 05:17
you can see https://github.com/mintty/mintty/issues/609 os.popen('chcp 65001 && ' + JAVA + ' -jar ' + CHECKSTYLE_JAR + ' -c ' + CHECKSTYLE_XML + ' "%s/%s"' % (COMMIT_TEMP_DIT, changed)).read()
msg281745 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-11-25 22:11
I don't see what this has to do with Python. Perhaps out Windows experts can, or can ask for more info. The last I knew, code page 65001 was buggy, but that is not a Python issue.
msg281751 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2016-11-26 02:11
For something as low level as os.popen you should read bytes and decode them manually with the correct encoding. If you use subprocess.Popen instead you can specify encoding="utf-8" in the call (on 3.6). Unfortunately, it's a very bad idea to rely on the default encoding for inter-process communication, but it's also impossible for us to change the defaults without breaking existing code unnecessarily. I'm not particularly aware of code-page 65001 itself being buggy, but there are certainly issues in the ANSI-mode console when you select it. On 3.6 we use the Unicode APIs directly for the console, but that has no impact on pipes or redirected handles, which is what this issue is encountering.
msg281758 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-11-26 07:39
So, does there appear to be a Python bug to fix or should this be closed?
msg281782 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2016-11-26 15:47
Unless the reporter comes back with more information, I'll assume it's not a bug.
History
Date User Action Args
2022-04-11 14:58:39 admin set github: 72944
2016-11-26 15:47:39 steve.dower set status: open -> closedresolution: not a bugmessages: + stage: resolved
2016-11-26 07:39:37 terry.reedy set messages: +
2016-11-26 02:11:16 steve.dower set messages: +
2016-11-25 22:11:56 terry.reedy set nosy: + terry.reedy, paul.moore, tim.golden, zach.ware, steve.dowermessages: + components: + Windows
2016-11-21 05:17:28 dontbugme create