[Python-Dev] IDLE and non-ASCII characters (original) (raw)

M.-A. Lemburg mal@lemburg.com
Sun, 13 May 2001 19:40:58 +0200

Martin von Loewis wrote:

Thanks to a bug report I got, I noticed for the first time that you cannot enter non-ASCII characters in IDLE anymore. Eg. at the shell prompt, you may get >>> s='��' UnicodeError: ASCII encoding error: ordinal not in range(128) Likewise, when trying to save a file that has non-ASCII characters, you get a traceback. Now, I think I understand all the causes of the problem (Tkinter returning Unicode objects, and so on). However, I'm curious whether anybody has proposals on how to deal with it. For saving text files, if Python had an encoding directive, things might be easier :-) For the shell prompt, I've no idea how to solve this best. So any suggestions are welcome.

I have a bug report assigned to myself which indicates similar problems with _tkinter and Tk/Tcl. There were other problem reports on the German Python mailing list going in the same direction too.

The basic problem seems to be that Tk/Tcl applies too much magic to the text widget contents in order to find out the used encoding and this can easily cause the whole encoding mechanism to fail.

A Tk/Tcl expert should really look into this and fix _tkinter.c to aid Tk/Tcl in not mixing up the encodings (e.g. it would probably be a good idea to recode Python 8bit-strings into whatever encoding Tk/Tcl assumes as default).

-- Marc-Andre Lemburg

Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/