[Python-Dev] Dynamic module namspaces (original) (raw)

Andrew Bennetts andrew-pythondev at puzzling.org
Mon Jul 17 07:15:44 CEST 2006

On Sat, Jul 15, 2006 at 03:38:04PM -0300, Johan Dahlin wrote:

In an effort to reduce the memory usage used by GTK+ applications written in python I've recently added a feature that allows attributes to be lazy loaded in a module namespace. The gtk python module contains quite a few attributes (around 850) of which many are classes or interfaces (150+)

Have you seen the "demandload" hack that Mercurial uses? You can find it here: http://selenic.com/repo/hg?f=cb4715847a81;file=mercurial/demandload.py

You can see an example use of it here: http://selenic.com/repo/hg?f=d276571f2c4b;file=mercurial/commands.py

The advantage for an interactive command line tool isn't so much memory consumption as speed. Why waste hundreds of milliseconds importing code that isn't used? There's an experimental branch to use the same demandload code in bzr, the reported results are "400ms for 'bzr rocks' down to 100ms, 'bzr root' from 400ms => 200ms, etc." (according to http://permalink.gmane.org/gmane.comp.version-control.bazaar-ng.general/13967)

Over half the runtime wasted on importing unused code! There's a definite need for a nice solution to this, and it should be included in the standard batteries that come with Python.

If we can address related problems at the same time, like emitting deprecation warnings for accessing certain module attributes, then even better!

-Andrew.

More information about the Python-Dev mailing list