[Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces (original) (raw)

Simon Cross hodgestar+pythondev at gmail.com
Fri Apr 24 12:39:15 CEST 2009

On Fri, Apr 24, 2009 at 12:04 PM, Glenn Linderman <glenn at nevcal.com> wrote:

The goal of Unicode users everywhere is to use Unicode for everything, no?  After all, all "real" file should have Unicode based names, and the only proper byte sequences that should exist are UTF-8 encoding Unicode bytes.  (Cheek to tongue: Get out of here!)

Humour aside :), the expectation that filenames are Unicode data simply doesn't agree with the reality of POSIX file systems.  I think an approach similar to that adopted by glib [1] could work -- i.e. use the bytes API and provide some tools to assist application developers in converting them to and from Unicode strings (these tools are then where all the guess work about what encoding to use can live).

[1] http://library.gnome.org/devel/glib/stable/glib-Character-Set-Conversion.html

Schiavo Simon

More information about the Python-Dev mailing list