[Python-Dev] unifying str and unicode (original) (raw)

M.-A. Lemburg mal at egenix.com
Tue Oct 4 01:35:57 CEST 2005

Martin Blais wrote:

On 10/3/05, Antoine Pitrou <solipsis at pitrou.net> wrote:

If that's how things were designed, then Python's entire standard brary (not to mention third-party libraries) is not "unicode safe" - to quote your own words - since many functions may return 8-bit strings containing non-ascii characters.

huh? first you talk about functions that convert unicode strings to 8-bit strings, now you talk about functions that return raw 8-bit strings? Are you deliberately missing the argument? And can't you understand that conversions are problematic in both directions (str -> unicode /and/ unicode -> str)? Both directions are a problem. Just a note: it's not so much the conversions that I find problematic, but rather the implicit nature of the conversions (combined with the fact that they may fail). In addition to being difficult to track down, these implicit conversions may be costing processing time as well.

We've already pointed you to a solution which you might want to use. Why don't you just try it ?

BTW, if you want to read up on all the reasons why Unicode was done the way it was, have a look at:

http://www.python.org/peps/pep-0100.html

and read up in the python-dev archives:

http://mail.python.org/pipermail/python-dev/2000-March/thread.html

and the next months after the initial checkin.

From what I've read on the web about the Python Unicode implementation we have one of the better ones compared to other languages implementations and their choices and design decisions.

None of them is perfect, but that's seems to be an inherent problem with Unicode no matter how you try to approach it - even more so, if you are trying to add it to a language that has used ordinary C strings for text from day 1.

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Sep 30 2005)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

More information about the Python-Dev mailing list