[Python-Dev] zipimport & import hooks (original) (raw)

Kevin Jacobs jacobs@penguin.theopalgroup.com
Fri, 6 Dec 2002 08:32:21 -0500 (EST)

On Fri, 6 Dec 2002, Fredrik Lundh wrote:

Just van Rossum wrote: > It seems the majority doesn't like non-strings on sys.path, which to me is > understandable, especially if you take PYTHONPATH into consideration. Let's try > to focus on that.

Has anyone presented any hard data, or are people just guessing? As far as I can tell, I'm the only one here who has posted fragile code from a real shipping product -- and I still believe it's a bogus argument.

I've reviews several hundred thousand lines of code in our repository, and here is the frequency breakdown of 588 usages of sys.path:

84%: sys.path.insert(0, package_path) 10%: sys.path.append(package_path) 2%: sys.path.extend(path_list) 1%: str(package_path) in sys.path <1%: random debugging/unit test code that prints sys.path or compares sys.path with a previous copy of sys.path.

The remaining ~1% use this fairly hairy idiom to implement a plug-in system:

in a package init.py

...

for filename in os.listdir(path[0]): if not filename.endswith('.py') or filename.startswith('__'): continue

name = filename[:-3]

plugin = __import__('%s.%s' % (base,name), None, None, [name])

# ...do something with plugin...

Clearly, the above code can be redone in a much nicer way that doesn't depend on path, the qualified base package name, and import.

Other than the above hard numbers, I have only some vague user-feelings to add to the discussion based on a poll of our in-house developers:

  1. They all are very convinced that

    sys.path.append('/usr/path/stuff.zip')

    should work just like

    mkdir /usr/path/stuff cd /usr/path/stuff unzip -d stuff.zip

    sys.path.append('/usr/path/stuff')

    Any other proposal that magically ignores the zip file name is right out, even if it makes it marginally more challenging to wrap the python standard library in a zip file.

  2. They don't care if Python stores objects, strings, or bananas in sys.path, so long as

    sys.path=map(str,sys.path)

    results in a human-readable path and does not change how imports occur.

  3. They all want ZIP archives to be a builtin feature, but don't care if they have to write an extra line of code to register a TAR/BZ2/whatever import hook.

  4. They don't care if a small amount of code breaks. It is trivial to notice when it does break and easy to fix.

Note that these comments are a result of a 15 minute discussion among a very mix bag of programmers. Some have been with Python for many years, others are in their first week of learning. Some come from primarily one of Java/C/C++/VB/Smalltalk/Scheme/Pascal backgrounds, others are extreme language polyglots. So, given the amount of discussion on python-dev, I was surprised how quickly the group converged upon the previous criteria with virtual unanimity. The only dissenting voice was our lone .NET fanatic who wouldn't shut up about how .NET assemblies were by far the superior way of doing things. We all have long since learned to ignore him.

Hope this helps, -Kevin

-- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com