[Python-Dev] bytes (original) (raw)

[Python-Dev] bytes / unicode

Nick Coghlan ncoghlan at gmail.com
Mon Jun 21 14:51:03 CEST 2010

On Mon, Jun 21, 2010 at 12:30 PM, P.J. Eby <pje at telecommunity.com> wrote:

I also find it weird that there seem to be two camps on this subject, one of which claims that All Is Well And There Is No Problem -- but I do not recall seeing anyone who was in the "What do I do; this doesn't seem ready" camp who switched sides and took the time to write down what made them realize that they were wrong about there being a problem, and what steps they had to take.  The existence of one or more such documents would certainly ease my mind, and I imagine that of other people who are less waiting for others' libraries, than for the stdlib (and/or language) itself to settle.

(Or more precisely, for it to be SEEN to have settled.)

I don't know that the "all is well" camp actually exists. The camp that I do see existing is the one that says "without a bug report, inconsistencies in the standard library's unicode handling won't get fixed".

The issues picked up by the regression test suite have already been dealt with, but that suite is unfortunately far from comprehensive. Just like a lot of Python code that is out there, the standard library isn't immune to the poor coding practices that were permitted by the blurry lines between text and octet streams in 2.x.

It may be that there are places where we need to rewrite standard library algorithms to be bytes/str neutral (e.g. by using length one slices instead of indexing). It may be that there are more APIs that need to grow "encoding" keyword arguments that they then pass on to the functions they call or use to convert str arguments to bytes (or vice-versa). But without people trying to port affected libraries and reporting bugs when they find issues, the situation isn't going to improve.

Now, if these bugs are already being reported against 3.1 and just aren't getting fixed, that's a completely different story...

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list