[Python-Dev] New relative import issue (original) (raw)

Phillip J. Eby pje at telecommunity.com
Fri Sep 22 03:40:57 CEST 2006

At 12:40 PM 9/22/2006 +1200, Greg Ewing wrote:

Guido van Rossum wrote:

> While I agree with your idea(l), I don't think that's what Greg meant. > He clearly say "sys.path should not exist at all". Refining that a bit, I don't think there should be a single sys.path for the whole program -- more like each module having its own sys.path. And, at least in most cases, its contents should be set up from static information that exists outside the program, established when the module is installed.

Now I'm a little bit more in agreement with you, but not by much. :)

In the Java world, the OSGi framework (which is a big inspiration for many aspects of setuptools) effectively has a sys.path per installed project (modulo certain issues for dynamic imports, which I'll get into more below).

And, when Bob Ippolito and I were first designing the egg runtime system, we considered using some hacks to do the same thing, such that you actually could have more than one version of a package present and being used by two other packages that require it.

However, we eventually tossed it as a YAGNI; dependency resolution is too damn hard already without also having to deal with crud like C extensions only being able to be loaded once, etc., in addition to the necessary import hackery.

So, in principle, it's a good idea. In practice, I don't think it can be achieved on the CPython platform. You need something like a Jython or IronPython or a PyPy+ctypes-ish platform, where all the C is effectively abstracted behind some barrier that prevents the module problem from occurring, and you could in principle load the modules in different interpreters (or "class loaders" in the Java context).

Amusingly, this is one of the few instances where every Python except CPython is probably in a better position to implement the feature... :)

One reason for this is the lack of any absolute notion of a "program". What is a program on one level can be part of a larger program on another level. For example, a module with test code that is run when it's invoked as a main script. Sometimes it's a program of its own, other times it's not. And it shouldn't matter whether it's a program or not when it comes to being able to find other modules that it needs to import. So using a piece of program-wide shared state for this seems wrong.

An interesting point about this is that it coincidentally solves the problem of dynamic interpretation of meta-syntactic features. That is, if there is a static way to know what modules are accessible to the module, then the resolution of meta-syntax features like macros or custom statements is simpler than if a runtime resolution is required.

Of course, in all of this you're still ignoring the part where some modules may need to perform dynamic or "weak" imports. So at least some portion of a module's import path is dependent on the notion of the "current program".

(See the documentation for the "Importing" package at http://peak.telecommunity.com/DevCenter/Importing for an explanation of "weak" importing.)

More information about the Python-Dev mailing list