[Python-Dev] urllib.quote and unquote - Unicode issues (original) (raw)

Matt Giuca matt.giuca at gmail.com
Sun Jul 13 02:24:11 CEST 2008

This POV is way too browser-centric...

This is but one example. Note that I found web forms to be the least clear-cut example of choosing an encoding. Most of the time applications seem to be using UTF-8, and all the standards I have read are moving towards specifying UTF-8 (from being unspecified). I've never seen a standard specify or even recommend Latin-1.

Where web forms are concerned, basically setting the form accept-charset or the page charset is the maximum amount of control you have over the encoding. As you say, it can be encoded by another page or the user can override their settings. Then what can you do as the server? Nothing ... there's no way to predict the encoding. So you just handle the cases you have control over.

  1. Different cultures do not choose necessarily between latin-1 and utf-8.

    They deal more with things like, say KOI8-R or Big5.

Exactly. This is exactly my point - Latin-1 is arbitrary from a standards point of view. It's just one of the many legacy encodings we'd like to forget. The UTFs are the only options which support all languages, and UTF-8 is the only ASCII-compatible (and therefore URI-compatible) encoding. So we should aim to support that as the default.

Besides all that and without any offense: "most proper" and "should do" and

the implication that all web browsers behave the same way are not a good location to argue from when talking about implementing a standard ;)

I agree. However if there was a proper standard we wouldn't have to argue! "Most proper" and "should do" is the most confident we can be when dealing with this standard, as there is no correct encoding.

Does anyone have a suggestion which will be more compatible with the rest of the world than allowing the user to select an encoding, and defaulting to "utf-8"? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20080713/d022d9f6/attachment.htm>

More information about the Python-Dev mailing list