[Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys (original) (raw)

M.-A. Lemburg mal at egenix.com
Tue Aug 8 17:08:18 CEST 2006

Martin v. Löwis wrote:

M.-A. Lemburg schrieb:

Failure to decode a string doesn't imply inequality. If the failure is "these bytes don't have a meaningful character interpretation", then the bytes are clearly not equal to some character string.

It implies that the programmer needs to step in and correct the problem by making an explicit and conscious decision. There is no problem to correct. The strings are inequal.

If the programmer writes:

x = 'äöü' y = u'äöü' ... if x == y: do_something()

then he clearly has had the intention to compare two character strings.

Now, if what you were saying were true, then the above would simply continue to work without raising an exception, possibly causing the application to return wrong results.

With the exception, the programmer will have a chance to correct the problem (in this case, probably a forgotten u-prefix) and also be safe in not having the application produce wrong data - something that's usually hard to detect, debug and, more importantly, can have effects which are a lot worse than a failing application.

Note that we are not discussing changing the behavior of the eq comparison between strings and Unicode, since this has always been to raise exceptions in case the automatic propagation fails.

The discussion is about silencing exceptions in the dict lookup mechanism - something which used to happen and now no longer is done.

Since this behavior is an implementation detail of the dictionary implementation, users perceive this change as random exceptions occurring in their application.

While these exceptions do hint at programming errors (the main reason for no longer silencing them), the particular case in the dict implementation requires some extra thought.

I've suggested to go about this in a slightly more user-friendly way, namely by giving a warning instead of raising an exception in Python 2.5 and then going for the exception in Python 2.6.

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Aug 08 2006)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

More information about the Python-Dev mailing list