[Python-Dev] Re: PEP 263 - default encoding (original) (raw)

Atsuo Ishimoto ishimoto@gembook.org
Wed, 13 Mar 2002 23:13:41 +0900

Hello,

I am a Japanese who sent a request to MAL. Let me explain about some issues.

  1. I didn't request to make sys.defaultencoding() as default. This is a part of the mail I sent you.

2. "native encoding" specifier

A special encoding name such as "native" or "defaultencoding" that means "to obey sys.defaultencoding()" may helpful. Here in Japan, we use two character encodings, ShiftJIS on Windows and EUC-JP on various Unix. When we move Python source files among these platforms, we usually have to convert the file encoding of the source files (just like as you convert CR+LF to LF when you move text files from Windows to Unix). Not only tools for manual encoding conversion, we also have a patched version of CVS to do it automatically. However, if we fix a file encoding by a magic comment, the task of encoding conversion will be complicated, because we should scan source files and update the magic comment when moving the files from one platform to the other. If we can use a "defaultencoding" encoding that the actual meaning varies among platforms, we can safely convert the file encoding without editing the magic comment, so that we can use common encoding conversion tools such as nkf(1), ack(1) and the patched version of CVS.

What I wanted is "special encoding name" that refers to sys.defaultencoding(). But, as I told to MAL, I changed my mind because I found I can create new special codec to do this, and it's pretty easy.

I'm sorry for late for sending mail about misapprehension, but I'm not on this list and I didn't know this until today. It is easier for me and other Japanese to join discussion if you move this topic to i18n-sig. (And if you can bear my English. The mail I sent to MAL was translated to English by Tamito, an author of the JapaneseCodecs).

About default encoding, I agree with ascii.

  1. Code conversion is necessary for Japanese programmers.

I think Python core don't have to mention about Japanese specific issues, because PEP 263 is general enough to support our encodings.

But please don't think that Japanese people are happy with utf-8. Suzuki wrote that I told we don't need code conversion, but this is not true. I don't know why he wrote so, and most of us needs native encodings such as ShiftJIS or EUC-JP.

Regards,

Atsuo Ishimoto ishimoto@gembook.org Homepage:http://www.gembook.jp