[Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3? (original) (raw)
M.-A. Lemburg mal at egenix.com
Wed Jun 29 10🔞59 CEST 2011
- Previous message: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?
- Next message: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Victor Stinner wrote:
Le mardi 28 juin 2011 à 16:02 +0200, M.-A. Lemburg a écrit :
How about a more radical change: have open() in Py3 default to opening the file in binary mode, if no encoding is given (even if the mode doesn't include 'b') ? I tried your suggested change: Python doesn't start.
No surprise there: it's an incompatible change, but one that undoes a wart introduced in the Py3 transition. Guessing encodings should be avoided whenever possible.
sysconfig uses the implicit locale encoding to read sysconfig.cfg, the Makefile and pyconfig.h. I think that it is correct to use the locale encoding for Makefile and pyconfig.h, but maybe not for sysconfig.cfg.
Python require more changes just to run "make". I was able to run "make" by using encoding='utf-8' in various functions (of distutils and setup.py). I didn't try the test suite, I expect too many failures.
This demonstrates that Python's stdlib is still not being explicit about the encoding issues. I suppose that things just happen to work because we mostly use ASCII files for configuration and setup.
--
Then I tried my suggestion (use "utf-8" by default): Python starts correctly, I can build it (run "make") and... the full test suite pass without any change. (I'm testing on Linux, my locale encoding is UTF-8.)
I bet it would also with "ascii" in most cases. Which then just means that the Python build process and test suite is not a good test case for choosing a default encoding.
Linux is also a poor test candidate for this, since most user setups will use UTF-8 as locale encoding. Windows, OTOH, uses all sorts of code page encodings (usually not UTF-8), so you are likely to hit the real problem cases a lot easier.
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Jun 29 2011)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
- Previous message: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?
- Next message: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]