[Python-Dev] Python install layout and the PATH on win32 (Rationale part 1: Regularizing the layout) (original) (raw)

M.-A. Lemburg mal at egenix.com
Fri Mar 23 11:40:47 CET 2012

VanL wrote:

As this has been brought up a couple times in this subthread, I figured that I would lay out the rationale here.

There are two proposals on the table: 1) Regularize the install layout, and 2) move the python binary to the binaries directory. This email will deal with the first, and a second email will deal with the second. 1) Regularizing the install layout: One of Python's strengths is its cross-platform appeal. Carefully- written Python programs are frequently portable between operating systems and Python implementations with very few changes. Over the years, substantial effort has been put into maintaining platform parity and providing consistent interfaces to available functionality, even when different underlying implementations are necessary (such as with ntpath and posixpath). One place where Python is unnecessarily different, however, is in the layout and organization of the Python environment. This is most visible in the name of the directory for binaries on the Windows platform ("Scripts") versus the name of the directory for binaries on every other platform ("bin"), but a full listing of the layouts shows substantial differences in layout and capitalization across platforms. Sometimes the include is capitalized ("Include"), and sometimes not; and the python version may or may not be included in the path to the standard library or not. This may seem like a harmless inconsistency, and if that were all it was, I wouldn't care. (That said, cross-platform consistency is its own good). But it becomes a real pain when combined with tools like virtualenv or the new pyvenv to create cross-platform development environments. In particular, I regularly do development on both Windows and a Mac, and then deploy on Linux. I do this in virtualenvs, so that I have a controlled and regular environment. I keep them in sync using source control. The problem comes when I have executable scripts that I want to include in my dvcs - I can't have it in the obvious place - the binaries directory - because *the name of the directory changes when you move between platforms.* More concretely, I can't hg add "Scripts/runner.py? on my windows environment (where it is put in the PATH by virtualenv) and thendo a pull on Mac or Linux and have it end up properly in "bin/runner.py" which is the correct PATH for those platforms. This applies anytime there are executable scripts that you want to manage using source control across platforms. Django projects regularly have these, and I suspect we will be seeing more of this with the new "project" support in virtualenvwrapper. While a few people have wondered why I would want this -- hopefully answered above -- I have not heard any opposition to this part of the proposal. This first proposal is just to make the names of the directories match across platforms. There are six keys defined in the installer files (sysconfig.cfg and distutils.command.install): 'stdlib', 'purelib', 'platlib', 'headers', 'scripts', and 'data'. Currently on Windows, there are two different layouts defined: 'nt': { 'stdlib': '{base}/Lib', 'platstdlib': '{base}/Lib', 'purelib': '{base}/Lib/site-packages', 'platlib': '{base}/Lib/site-packages', 'include': '{base}/Include', 'platinclude': '{base}/Include', 'scripts': '{base}/Scripts', 'data' : '{base}', }, 'ntuser': { 'stdlib': '{userbase}/Python{pyversionnodot}', 'platstdlib': '{userbase}/Python{pyversionnodot}', 'purelib': '{userbase}/Python{pyversionnodot}/site-packages', 'platlib': '{userbase}/Python{pyversionnodot}/site-packages', 'include': '{userbase}/Python{pyversionnodot}/Include', 'scripts': '{userbase}/Scripts', 'data' : '{userbase}', },

The proposal is to make all the layouts change to: 'nt': { 'stdlib': '{base}/lib', 'platstdlib': '{base}/lib', 'purelib': '{base}/lib/site-packages', 'platlib': '{base}/lib/site-packages', 'include': '{base}/include', 'platinclude': '{base}/include', 'scripts': '{base}/bin', 'data' : '{base}', }, The change here is that 'Scripts' will change to 'bin' and the capitalization will be removed. Also, "user installs" of Python will have the same internal layout as "system installs" of Python. This will also, not coincidentally, match the install layout for posix, at least with regard to the 'bin', 'lib', and 'include' directories. Again, I have not heard anyone objecting to this part of the proposal as it is laid out here. (Paul had a concern with the lib directory earlier, but he said he was ok with the above). Please let me know if you have any problems or concerns with this part 1.

Since userbase will usually be a single directory in the home dir of a user, the above would loose the possibility to support multiple Python versions in that directory.

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Mar 23 2012)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

2012-04-03: Python Meeting Duesseldorf 11 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/

More information about the Python-Dev mailing list