[Python-Dev] PEP 420 - dynamic path computation is missing rationale (original) (raw)

Eric V. Smith eric at trueblade.com
Tue May 22 16:51:22 CEST 2012

On 05/21/2012 07:25 PM, Nick Coghlan wrote:

As a simple example to back up PJE's explanation, consider: 1. encodings becomes a namespace package 2. It sometimes gets imported during interpreter startup to initialise the standard io streams 3. An application modifies sys.path after startup and wants to contribute additional encodings

Searching the entire parent path for new portions on every import would be needlessly slow. Not recognising new portions would be needlessly confusing for users. In our simple case above, the application would fail if the io initialisation accessed the encodings package, but work if it did not (e.g. when all streams are utf-8). PEP 420 splits the difference via an automatically invalidated cache: when you iterate over a namespace package path object, it rescans the parent path for new portions if and only if the contents of the parent path have changed since the previous scan.

That seems like a pretty convincing example to me.

Personally I'm +1 on putting dynamic computation into the PEP, at least for top-level namespace packages, and probably for all namespace packages.

The code is not very large or complicated, and with the proposed removal of the restriction that sys.path cannot be replaced, I think it behaves well.

But Guido can decide against it without hurting my feelings.

Eric.

P.S.: Here's the current code in the pep-420 branch. This code still has the restriction that sys.path (or parent_path in general) can't be replaced. I'll fix that if we decide to keep the feature.

class _NamespacePath: def init(self, name, path, parent_path, path_finder): self._name = name self._path = path self._parent_path = parent_path self._last_parent_path = tuple(parent_path) self._path_finder = path_finder

def _recalculate(self):
    # If _parent_path has changed, recalculate _path
    parent_path = tuple(self._parent_path)     # Make a copy
    if parent_path != self._last_parent_path:
        loader, new_path = self._path_finder(self._name, parent_path)
        # Note that no changes are made if a loader is returned, but we
        #  do remember the new parent path
        if loader is None:
            self._path = new_path
        self._last_parent_path = parent_path   # Save the copy
    return self._path

def __iter__(self):
    return iter(self._recalculate())

def __len__(self):
    return len(self._recalculate())

def __repr__(self):
    return "_NamespacePath" + repr((self._path, self._parent_path))

def __contains__(self, item):
    return item in self._recalculate()

More information about the Python-Dev mailing list