[Python-Dev] PEP 420 - dynamic path computation is missing rationale (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Wed May 23 03:58:28 CEST 2012

On Wed, May 23, 2012 at 10:40 AM, Eric V. Smith <eric at trueblade.com> wrote:

On 5/22/2012 2:37 PM, Guido van Rossum wrote:

Okay, I've been convinced that keeping the dynamic path feature is a good idea. I am really looking forward to seeing the rationale added to the PEP -- that's pretty much the last thing on my list that made me hesitate. I'll leave the details of exactly how the parent path is referenced up to the implementation team (several good points were made), as long as the restriction that sys.path must be modified in place is lifted. I've updated the PEP. Let me know how it looks. I have not updated the implementation yet. I'm not exactly sure how I'm going to convert from a path list of unknown origin to ('sys', 'path') or ('foo', 'path'). I'll look at it later tonight to see if it's possible. I'm hoping it doesn't require major surgery to importlib.bootstrap.

If you wanted to do this without changing the sys.meta_path hook API, you'd have to pass an object to find_module() that did the dynamic lookup of the value in obj.iter. Something like:

class _LazyPath:
    def __init__(self, modname, attribute):
        self.modname = modname
        self.attribute = attribute
    def __iter__(self):
        return iter(getattr(sys.module[self.modname], self.attribute))

A potentially cleaner alternative to consider is tweaking the find_loader API spec so that it gets used at the meta path level as well as at the path hooks level and is handed a callable that dynamically retrieves the path rather than a direct reference to the path itself.

The full signature of find_loader would then become:

def find_loader(fullname, get_path=None):
    # fullname as for find_module
    # When get_path is None, it means the finder is being called

as a path hook and # should use the specific path entry passed to init # In this case, namespace package portions are returned as (None, portions) # Otherwise, the finder is being called as a meta_path hook and get_path() will return the relevant path # Any namespace packages are then returned as (loader, portions)

There are two major consequences of this latter approach:

That second consequence is rather appealing: it means you'd be able to implement an almost complete walk of a package hierarchy without having to import anything (although you would miss old-style namespace packages and any other packages that alter their own path in init, so you may still want to load packages to make sure you found everything. You could definitively answer the "is this a package or not?" question without running any code, though).

The first consequence is also appealing, since the find_module() name is more than a little misleading. The "find_module" name strongly suggests that the method is expected to return a module object, and that's just wrong - you actually find a loader, then you use that to load the module.

Cheers, Nick.

-- Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-Dev mailing list