[Python-Dev] PEP: Post import hooks (original) (raw)

Christian Heimes lists at cheimes.de
Thu Jan 10 21:40:05 CET 2008

Phillip J. Eby wrote: [...]

There's also one twist that I haven't sorted out yet: "Importing" guarantees that a parent module 'foo' will have a 'bar' attribute for the 'foo.bar' module, if 'foo.bar' is lazy. It does this by registering a callback, ideally before any other callback is registered for 'foo' or 'foo.bar' that would look at 'foo.bar'. I don't see how to maintain this condition in a world where import callbacks can be registered independently.

I've moved the PyImport_NotifyModuleLoaded() call to import_submodule(). It (should) guarantee that the hooks for a parent package is called before the hooks for its children are called. I've analyzed the code carefully enough to be sure but all unit test results are on my side.

On other words "import a.b.c" fires the hooks for "a", then "a.b" and at last "a.b.c".

I could also modify imp.notify_module_loaded to accepts the module name as string ("a.b.c."). If the module is provided by name (e.g. "a.b.c.") rather than by object it makes sure that the hooks for "a", "a.b" and "a.b.c" are called in the right order.

Bleah. All of the above isn't really a good explanation of the problem. Let me try to simplify it:

* Lazy importing needs to guarantee that foo.bar = sys.modules['foo.bar'], when callbacks for 'foo.bar' execute (in case they refer to foo.bar) * To do this, it registers a callback that sets foo.bar = sys.modules['foo.bar'], and does not actually register any foo.bar callbacks until 'foo' is really imported (and thus foo.bar gets set by that callback)

Would the modification fulfill your needs if imp.notify_module_loaded("foo.bar.baz") call the hooks for "foo", "foo.bar" and "foo.bar.baz" in that order?

In the case of the PEP, it's harder for me to figure out what happens, because you might not have any lazy modules around, and the foo.bar issue would then not come up. You also have the possibility of a problem where a lazy import callback occurs in 3rd party code, while callbacks are occurring from the import machinery. (Which means that the notification API should probably set the hooks entry to None while it's running, so that if it's called from inside a hook, it will not double-run the hooks, and new hooks registered while hooks are running will get run immediately as they are encountered, instead of getting added to the list.)

The initial design used to set the hooks to None after the hooks were called. I removed code yesterday because I thought it's not required. Today I've re-added the checks for Py_None. I'm not setting the hooks to Py_None before the hook are called.

Christian

More information about the Python-Dev mailing list