[Python-ideas] changing sys.stdout encoding (original) (raw)

MRAB python at mrabarnett.plus.com
Wed Jun 6 01:56:55 CEST 2012

On 06/06/2012 00:34, Victor Stinner wrote:

2012/6/5 Rurpy<rurpy at yahoo.com>:

In my first foray into Python3 I've encountered this problem: I work in a multi-language environment. I've written a number of tools, mostly command-line, that generate output on stdout. Because these tools and their output are used by various people in varying environments, the tools all have an --encoding option to provide output that meets the needs and preferences of the output's ultimate consumers. What happens if the specified encoding is different than the encoding of the console? Mojibake? If the output is used as in the input of another program, does the other program use the same encoding? In my experience, using an encoding different than the locale encoding for input/output (stdout, environment variables, command line arguments, etc.) causes various issues. So I'm curious of your use cases.

In converting them to Python3, I found the best (if not very pleasant) way to do this in Python3 was to put something like this near the top of each tool[*1]:

import codecs sys.stdout = codecs.getwriter(opts.encoding)(sys.stdout.buffer) In Python 3, you should use io.TextIOWrapper instead of codecs.StreamWriter. It's more efficient and has less bugs. What I want to be able to put there instead is: sys.stdout.setencoding (opts.encoding) I don't think that your use case merit a new method on io.TextIOWrapper: replacing sys.stdout does work and should be used instead. TextIOWrapper is generic and your use case if specific to sys.std* streams. It would be surprising to change the encoding of an arbitrary file after it is opened. At least, I don't see the use case. [snip]

And if you do want multiple encodings in a file, it's clearer to open the file as binary and then explicitly encode to bytes and write that to the file.

More information about the Python-ideas mailing list