[Python-Dev] codecs.oen [was: PEP 385: the eol-type issue] (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Mon Aug 10 15:45:32 CEST 2009

Jim Jewett <jimjjewett gmail.com> writes:

In python 3, why does codecs.open even still exist?

I don't remember anyone proposing to deprecate it, so I suppose that's the (social) reason.

So at this point, are there any differences beyond:

(c) The built-in open is probably a little more featureful, especially when it comes to seek() and tell().

(b) The codecs version is much slower, because it hasn't seen the optimization effort.

By the way, the built-in open would also benefit from an optimization of codecs.py's IncrementalEncoder classes: they are just thin Python wrappers around C function calls, and the overhead of calling a Python method is very significant when doing a lot of small unicode writes with a non-optimized codec (a couple of dominant codecs have been optimized by means of internal shortcuts bypassing codecs.py: latin-1, utf-8, utf-16).

Regards

Antoine.

(a) The builtin open doesn't work on multi-byte line-endings other than the multi-character CRLF. (In other words, it goes by the traditional Operating System conventions developed when a char was a byte, but the Unicode standard allows for a few more possibilities, which are currently rare in practice.)

More information about the Python-Dev mailing list