[Python-Dev] Proposal for virtualenv functionality in Python (original) (raw)

Ian Bicking ianb at colorstudy.com
Fri Feb 19 19:49:23 CET 2010

This is a proto-proposal for including some functionality from virtualenv in Python itself. I'm not entirely confident about what I'm proposing, so it's not really PEP-ready, but I wanted to get feedback...

First, a bit about how virtualenv works (this will use Linux conventions; Windows and some Mac installations are slightly different):

There are some tweaks to this that could be made, but I believe virtualenv basically does things The Right Way. By setting sys.prefix All Tools Work (there are some virtualenv alternatives that do isolation without setting sys.prefix, but they typically break more often than virtualenv, or only support a limited number of workflows). Also by using a distinct interpreter (~/env/bin/python) it works fairly consistently and reliably compared to techniques like an environmental variable. The one serious alternative is what buildout (and virtualenv --relocatable) does, which is to use the system Python and change the path at the beginning of all scripts (it requires its own installer to accomplish this consistently).

But virtualenv is kind of a hack, and I believe with a little support from Python this could be avoided. virtualenv can continue to exist to support the equivalent workflows on earlier versions of Python, but it would not exist (or would become much much simpler) on further Python versions.

The specific parts of virtualenv that are a hack that I would like to replace with built-in functionality:

I am not sure what the best way to do this is, but I will offer at least one suggestion (other suggestions welcome):

In my (proto-)proposal, a new binary pythonv is created. This is slightly like pythonw.exe, which provides a Python interpreter on Windows which doesn't open a new window. This binary is primarily for creating new environments. It doesn't even need to be on $PATH, so it would be largely invisible to people unless they use it.

If you symlink pythonv to a new location, it will effect sys.prefix (currently sys.prefix is calculated after dereferencing the symlink).

Additionally, the binary will look for a configuration file. I'm not sure where this file should go; perhaps directly alongside the binary, or in some location based on sys.prefix.

The configuration file would be a simple set of assignments; some I might imagine:

I got some feedback from the Debian/Ubuntu maintainer that he would like functionality that might be like this; for instance, if you have /usr/bin/python2.6 and /usr/bin/python2.6-dbg, he'd like them to work slightly different (e.g., /usr/bin/python2.6-dbg would look in a different place for libraries). So the configuration file location should be based on sys.prefix and the name of the binary itself (e.g., /usr/lib/python2.6/python-config-dbg.conf). I have no strong opinion on the location of the file itself, only that it can be specific to the directory and name of the interpreter.

In addition to all this, I think sys would grow another prefixy value, e.g., sys.build_prefix, that points to the place where Python was actually built (virtualenv calls this sys.real_prefix, but that's not a very good name). Some code, especially in distutils, might need to be aware of this to compile extensions properly (we can be somewhat aware of these cases by looking at places where virtualenv already has problems compiling extensions).

Some people have argued for something like sys.prefixes, a list of locations you might look at, which would allow a kind of nesting of these environments (where sys.prefixes[-1] == sys.prefix; or maybe reversed). Personally this seems like it would be hard to keep mental track of this, but I can understand the purpose -- you could for instance create a kind of template prefix that has most of what you want installed in it, then create sub-environments that contain for instance an actual application, or a checkout (to test just one new piece of code).

I'm not sure how this should best work on Windows (without symlinks, and where things generally work differently), but I would hope if this idea is more visible that someone more opinionated than I would propose the appropriate analog on Windows.

-- Ian Bicking | http://blog.ianbicking.org | http://twitter.com/ianbicking -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20100219/49d99270/attachment.htm>

More information about the Python-Dev mailing list