[Python-Dev] PEP: Post import hooks (original) (raw)

Christian Heimes lists at cheimes.de
Tue Jan 15 22:14:20 CET 2008

Phillip J. Eby wrote:

whenimported('a.b')(funcab1) whenimported('a.b')(funcab2)

@whenimported('a') def funca1(modulea): whenimported('a.b')(funcab3) notifymodule('a.b') # <- this is here to foil trivial implementations whenimported('a')(funca2) notifymodule('a.b') This should produce the calling sequence: funca1, funca2, funcab1, funcab2, funcab3.

My implementation calls the hooks in the right order but I had to insert fake modules into sys.path.

    # insert the modules into sys.modules to fake a 3rd party import
    a = imp.new_module('a')
    ab = imp.new_module('a.b')
    a.b = ab
    sys.modules["a"] = a
    sys.modules["a.b"] = ab
    # notify
    imp.notify_module_loaded('a.b')

Otherwise the code fails to fetch the module objects from sys.path and an exception is raised.

1. notification is only done once for a given module, ever, even if the notification function is called more than once, even if it's called during notifications for that module

The latter is not yet implemented. notify_module('a.b') currently fails with a recursion limit exceeded. It's on my todo list.

2. notifications for a child module/package may not begin until the notifications for the parent package have begun

This is already implemented for notification by name but not for notification by module object. It's also on my todo list.

3. two registrations for the same module must always be invoked in the same order as they were registered, even if some of the registrations are done during notification.

I'm pretty but not yet absolute sure that my implementation guarantees the last invariant. I've to go through my code several times and play through use cases.

In order to implement these invariants, you will have to have a way to know whether notifications have been begun for a given module. In peak.util.imports, the module objects effectively keep track of this, although they don't have a specific flag. For the Python implementation, you could add a notified field to module objects, and implement the notify function thus:

That's a nice idea to fix the recursion issue with nested notifications.

Of course, notified would actually be a structure slot, rather than an attribute, so as to avoid any attribute lookup issues with module subtypes (e.g. lazy modules).

Sure, I can use a slot for PyModule and its subclasses. Unfortunately other implementations of lazy imports are adding a proxy object which is not a subclass of PyModule. I've to use an attribute if not isinstance(mod, moduletype).

The register function would simply append a hook to the entry in postimporthooks if it's not None, or call the hook otherwise.

My code queues up new hooks while a sequence of hooks is processed. It makes sure that hooks for a parent aren't called in the middle of a child's hook chain.

notification_in_progress = False queue = []

def register_hook(hook, name): if notification_in_progress: queue.append((hook, name)) return

hooks = sys.post_import_hook_register(name, None)

if hooks is None:
    module = sys.modules.get(name, None)
    if modules:
        sys.post_import_hook_register[name] = None
        hook(module)
    else:
        hooks = []
        sys.post_import_hook_register[name] = hooks

hooks.append(hook)

def notify_loaded(mod_or_name): notification_in_progress = True try: ... finally: notification_in_progress = False while queue: hook, name = queue.pop(0) hook(sys.modules[name])

With this implementation, I could make a version of peak.util.imports that did its own lazy modules, but used the base system for all the hooks.

Very good!

Christian

More information about the Python-Dev mailing list