[Python-Dev] PEP 405 (built-in virtualenv) status (original) (raw)

Carl Meyer carl at oddbird.net
Mon Mar 19 20🔞40 CET 2012

Hello Kristján,

On 03/19/2012 03:26 AM, Kristján Valur Jónsson wrote:

Hi Carl. I'm very interested in this work. At CCP we work heavily with virtual environments. Except that we don't use virtualenv because it is just a pain in the neck. We like to be able to run virtual python environments of various types as they arrive checked out of source control repositories, without actually "installing" anything. For some background, please see: http://blog.ccpgames.com/kristjan/2010/10/09/using-an-isolated-python-exe/. It's a rather quick read, actually.

The main issue for us is: How to prevent your local python.exe from reading environment variables and running some global site.py? There are a number of points raised in the above blog, please take a look at the "Musings" at the end.

Thanks, I read the blog post. I think there are some useful points there; I too find the startup sys.path behavior of Python a bit more complex and magical than I'd prefer (I'm sure it's grown organically over the years to address a variety of different needs and concerns, not all of which I understand or am even aware of).

I think there's one important (albeit odd and magical) bit of Python's current behavior that you are missing in your blog post. All of the initial sys.path directories are constructed relative to sys.prefix and sys.exec_prefix, and those values in turn are determined (if PYTHONHOME is not set), by walking up the filesystem tree from the location of the Python binary, looking for the existence of a file at the relative path "lib/pythonX.X/os.py" (or "Lib/os.py" on Windows). Python takes the existence of this file to mean that it's found the standard library, and sets sys.prefix accordingly. Thus, you can achieve reliable full isolation from any installed Python, with no need for environment variables, simply by placing a file (it can even be empty) at that relative location from the location of your Python binary. You will still get some default paths added on sys.path, but they will all be relative to your Python binary and thus presumably under your control; nothing from any other location will be on sys.path. I doubt you will find this solution satisfyingly elegant, but you might nonetheless find it practically useful.

The essence of PEP 405 is simply to provide a less magical way to achieve this same result, by locating a "pyvenv.cfg" file next to (or one directory up from) the Python binary.

The bulk of the work in PEP 405 is aimed towards a rather different goal from yours - to be able to share an installed Python's copy of the standard library among a number of virtual environments. This is the purpose of the "home" key in pyvenv.cfg and the added sys.base_prefix (which point to the Python installation whose standard library will be used). I think this serves a valuable and common use case, but I wonder if your use case couldn't also be served with a minor tweak to PEP 405. Currently it ignores a pyvenv.cfg file with no "home" key; instead, it could set sys.prefix and sys.base_prefix both to the location of that pyvenv.cfg. For most purposes, this would result in a broken Python (no standard library), but it might help you?

Beyond that possible tweak, while I certainly wouldn't oppose any effort to clean up / document / make-optional Python's startup sys.path-setting behavior, I think it's mostly orthogonal to PEP 405, and I don't think it would be helpful to expand the scope of PEP 405 to include that effort.

Carl

-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: OpenPGP digital signature URL: <http://mail.python.org/pipermail/python-dev/attachments/20120319/bc8dd11b/attachment.pgp>

More information about the Python-Dev mailing list