[Python-Dev] New codecs checked in (original) (raw)

M.-A. Lemburg mal at egenix.com
Tue Oct 25 11:17:58 CEST 2005

Martin v. Löwis wrote:

M.-A. Lemburg wrote:

I just left them in because I thought they wouldn't do any harm and might be useful in some applications.

Removing them where not directly needed by the codec would not be a problem. I think memory usage caused is measurable (I estimated 4KiB per dictionary). More importantly, people apparently currently change the dictionaries we provide and expect the codecs to automatically pick up the modified mappings. It would be better if the breakage is explicit (i.e. they get an AttributeError on the variable) instead of implicit (their changes to the mapping simply have no effect anymore).

Agreed. I've already checked in the changes, BTW.

KOI8-U is not available as mapping on ftp.unicode.org and I only recreated codecs from the mapping files available there.

I think we should come up with mapping tables for the additional codecs as well, and maintain them in the CVS. This also applies to things like rot13.

Agreed.

I'll rerun the creation with the above changes sometime this week.

I hope I can finish my encoding routine shortly, which again results in changes to the codecs (replacing the encoding dictionaries with other lookup tables).

Having seen the decode tables written as long Unicode string, I think that this may indeed also be a good solution for encoding - the major improvement here is that the parser and compiler will do the work of creating the table. At module load time, the .pyc file will only contain a long string which is very fast to create and load (unlike dictionaries which are set up dynamically at load time).

In general, it's better to do all the work up-front when creating the codecs, rather than having run-time code repeat these tasks over and over again.

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Oct 25 2005)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

More information about the Python-Dev mailing list