[Python-Dev] Freezing unicode codecs. (original) (raw)

M.-A. Lemburg mal@lemburg.com
Fri, 11 Aug 2000 10:04:48 +0200

Finn Bock wrote:

While porting the unicode API and the encoding modules to JPython I came across a problem which may also (or maybe not) exists in CPython. jpythonc is a compiler for jpython which try to track dependencies between modules in an attempt to detect which modules an application or applet uses. I have the impression that some of the freeze tools for CPython does something similar. A call to unicode("abc", "cp1250") and "abc".encode("cp1250") will cause the encoding.cp1250 module to be loaded as a side effect. The freeze tools will have a hard time figuring this out by scanning the python source. For JPython I'm leaning towards making it a requirement that the encodings must be loading explicit from somewhere in application. Adding import encoding.cp1250 somewhere in the application will allow jpythonc to include this python module in the frozen application. How does CPython solve this?

It doesn't. The design of the codec registry is such that it uses search functions which then locate and load the codecs. These search function can implement whatever scheme they desire for the lookup and also w/r to loading the codec, e.g. they could get the data from a ZIP archive.

This design was chosen to allow drop-in configuration of the Python codecs. Applications can easily add new codecs to the registry by registering a new search function (and without having to copy files into the encodings Lib subdir).

When it comes to making an application freezable, I'd suggest adding explicit imports to some freeze support module in the application. There are other occasions where this is needed too, e.g. for packages using lazy import of modules such as mx.DateTime.

This module would then make sure freeze.py finds the right modules to include in its output.

PS. The latest release of the JPython errata have full unicode support and includes the "sre" module and unicode codecs.

http://sourceforge.net/project/filelist.php?groupid=1842

Cool :-)

regards, finn

Python-Dev mailing list Python-Dev@python.org http://www.python.org/mailman/listinfo/python-dev

-- Marc-Andre Lemburg

Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/