[Python-Dev] PEP: Post import hooks (original) (raw)

Christian Heimes lists at cheimes.de
Wed Jan 16 02:28:15 CET 2008

Phillip J. Eby wrote:

At 10:14 PM 1/15/2008 +0100, Christian Heimes wrote:

My code queues up new hooks while a sequence of hooks is processed. It makes sure that hooks for a parent aren't called in the middle of a child's hook chain. Notice that that's not necessary with the notification algorithm I gave, since the list in postimporthooks suffices as a queue. So, just as in peak.util.imports, the registration code doesn't need to know whether callbacks are being run; it only needs to know whether they're finished.

Are you sure your proposed algorithm and output match for the test case? I'm confident I got it right in C but I'm getting a different output.

Without the extra imp.notify_module_loaded('a.b') in func_a1(mod)::

['func_a1', 'func_a2', 'func_ab1', 'func_ab2', 'func_ab3']

With the extra imp.notify_module_loaded('a.b') in func_a1(mod)::

['func_a1', 'func_ab1', 'func_ab2', 'func_ab3', 'func_a2']

I can't see how your implementation results in the first output when func_a1() calls the notification method.

Of course, both the notification and registration functions must hold the import lock to prevent a race condition where one thread adds a hook to the list after another thread has just finished iterating over it and is about to replace the list with None. At least, they have to if they're executing any Python code that might cause the GIL to be released. The callbacks will release the GIL, of course, but the registration code probably doesn't... well, it will if it calls the hook, and ISTM that the hooks should always execute with the import lock held, even if they're fired at registration.

I'm aware of the implications and my code already uses the lock. The PyImport_NotifyLoaded() method excepts to be called with the importer lock acquired. So I'm locking the importer lock in imp_notify_module_loaded(). The PyImport_RegisterPostImportHook() method does the locking before it accesses sys.modules and sys.post_import_hooks.

Christian

More information about the Python-Dev mailing list