[Python-Dev] Inconsistencies if locale and filesystem encodings are different (original) (raw)

Victor Stinner victor.stinner at haypocalc.com
Thu Oct 7 21:14:36 CEST 2010

Le jeudi 07 octobre 2010 18:35:09, M.-A. Lemburg a écrit :

Victor Stinner wrote: > Hi, > > A PYTHONFSENCODING environment variable was added to Python 3.2: issue > #8622. This variable introduces an inconstency because the filesystem > and the locale encodings can now be different. > > There are (at least) four issues related to this problem. We have 2 > choices to > > fix these issues: > (a) use the same encoding to encode and decode values (it can be > different > > for each issue) > > (b) remove PYTHONFSENCODING variable and raise an error if locale and > > filesystem encodings are different (ensure that both encodings are the > same) > > Even if choice (a) is not easy to implement, it is feasible and I already > wrote some patches. > > I don't understand how Python interact with other programs who ignore the > PYTHONFSENCODING environment variable. It's like Python uses its own > "locale". > > Choice (b) looks easy to implement, but... there is the problem of Mac OS > X. Mac OS X uses utf-8 encoding for the filesystem (and not the locale > encoding), whereas it looks like the locale encoding is used for the > command line arguments. See issue #4388 for more information. > > There is also maybe an useful usecase of the PYTHONFSENCODING, but I > don't remember which one :-)

You have to differentiate between the meaning of a file system encoding and the locale: A file system encoding defines how the applications interact with the file system. A locale defines how the user expects to interact with the application.

What is the encoding of the command line arguments? Locale or filesystem encoding? Is it different if an argument is a filename or a path?

-- Victor Stinner http://www.haypocalc.com/

More information about the Python-Dev mailing list