[Python-Dev] PEP 263 considered faulty (for some Japanese) (original) (raw)

Fredrik Lundh fredrik@pythonware.com
Wed, 13 Mar 2002 13:34:30 +0100

mal wrote:

The PEP says: """ The builtin compile() API will be enhanced to accept Unicode as input. 8-bit string input is subject to the standard procedure for encoding detection as decsribed above. """

so if you pass in a Unicode string, any "coding" declaration is ignored?

so the problem of decoding source code input to compile() is shifted into the application space.

in other words, something like this should work:

stream =3D make_http_request(...)
body =3D stream.read()
charset =3D stream.getheader("content-type", "charset")
if charset:
    body =3D unicode(body, charset)
code =3D compile(body, ...)