[Python-Dev] Encoding detection in the standard library? (original) (raw)

David Wolever wolever at cs.toronto.edu
Mon Apr 21 19:14:07 CEST 2008

On 21-Apr-08, at 12:44 PM, skip at pobox.com wrote:

David> Is there some sort of text encoding detection module is the David> standard library? And, if not, is there any reason not to add David> one? No, there's not. I suspect the fact that you can't correctly determine the encoding of a chunk of text 100% of the time mitigates against it. Sorry, I wasn't very clear what I was asking.

I was thinking about making an educated guess -- just like chardet
(http://chardet.feedparser.org/).

This is useful when you get a hunk of data which should be some
sort of intelligible text from the Big Scary Internet (say, a posted
web form or email message), and you want to do something useful with
it (say, search the content).

More information about the Python-Dev mailing list