[Python-Dev] PEP 540: Add a new UTF-8 mode (v2) (original) (raw)

Glenn Linderman v+python at g.nevcal.com
Wed Dec 6 01🔞06 EST 2017

On 12/5/2017 8:07 PM, INADA Naoki wrote:

Oh, revised version is really short!

And I have one worrying point. With UTF-8 mode, open()'s default encoding/error handler is UTF-8/surrogateescape. Containers are really growing. PyCharm supports Docker and many new Python developers use Docker instead of installing Python directly on their system, especially on Windows. And opening binary file without "b" option is very common mistake of new developers. If default error handler is surrogateescape, they lose a chance to notice their bug.

"b" mostly matters on Windows, correct? And Windows doesn't use C or POSIX locale, correct? And if these are correct, then is this an issue? And if so, why?

On the other hand, it helps some use cases when user want byte-transparent behavior, without modifying code to use "surrogateescape" explicitly.

Which is more important scenario? Anyone has opinion about it? Are there any rationals and use cases I missing? Regards, INADA Naoki <songofacandy at gmail.com>

On Wed, Dec 6, 2017 at 12:17 PM, INADA Naoki <songofacandy at gmail.com> wrote: I'm sorry about my laziness. I've very busy these months, but I'm back to OSS world from today.

While I should review carefully again, I think I'm close to accept PEP 540. * PEP 540 really helps containers and old Linux machines PEP 538 doesn't work. And containers is really important for these days. Many new Pythonistas who is not Linux experts start using containers. * In recent years, UTF-8 fixed many mojibakes. Now UnicodeError is more usability problem for many Python users. So I agree opt-out UTF-8 mode is better than opt-in on POSIX locale. I don't have enough time to read all mails in ML archive. So if someone have opposite opinion, please remind me by this weekend. Regards,

Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/v%2Bpython%40g.nevcal.com

-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20171205/8bf347bd/attachment-0001.html>

More information about the Python-Dev mailing list