[Python-Dev] Python-3 transition in Arch Linux (original) (raw)

Allan McRae allan at archlinux.org
Thu Nov 4 05:44:09 CET 2010

Hi,

While this is not strictly related to python development, I thought that developers of python might be interested in some of the lessons provided by this. So forgive me if this is really wrong for this list...

Recently Arch Linux did a big transition with respect to python. Now we support two python packages: "python" and "python2".

The "python" package will always contain the latest 3.x release and currently has /usr/bin/python3.1 with symlinks to /usr/bin/python3 and /usr/bin/python.

The "python2" package contains the latest from the "legacy" python-2.x branch and has /usr/bin/python2.7 with a symlink to /usr/bin/python2.

I really do not want to debate the sanity of pointing /usr/bin/python at python-3.x here, but it suffices to say that I am of the opinion that if python-3.x is really the future of python, then /usr/bin/python must eventually point to a 3.x version. Also, Arch Linux is very bleeding edge and we expect our users to be competent enough to deal with thing like this. According to #python, we are all idiots.... And I have been (figuratively) yelled at by a couple of Debian developers (which is incidentally the only major distro I found without a /usr/bin/python2 symlink).

Anyway, this transition was rather simple from a distribution point of view apart from the sheer number of packages involved. All our supported packages were rebuilt to work with that symlink layout and any "porting" software to use that layout was relatively simple. Most packages either detected the python2 symlink during the rebuild and just worked while others required some sort "export PYTHON=python2" or "--with-python=python2" or "python2 setup.py" or the like during the build.

Software packages tend to fall into three categories at roughly equal frequencies:

  1. packages that detected or were pointed at python2 and everything worked
  2. packages that detected or were pointed at python2 and partially worked
  3. packages that needed adjustment to work with the python2 symlink.

The second case was particularly interesting. These software would change some of their #! to point at the python2 symlink and leave the rest pointing at python. Note that python-2.7 itself falls into this category as many files in /usr/lib/python2.7 still have "#!/usr/bin/env python" even when installed with "make altinstall". I can not remember the exact details, but I recall that some of these files were installed with executable permissions which would be bad, but I need to look into this again now things have calmed down...

The packages that did not auto-detect and work with /usr/bin/python2 or /usr/bin/python2.7 mostly required a sed of their shebangs or a patch to any hardcoded /usr/bin/python paths so were easily fixed.

So that is something that python software developers could think about for the future. When someone configures a module using a particular version of python, then ALL shebangs need changed to use that version. And it is generally bad practice to hardcode /usr/bin/python into any application as you are never quite sure which version you are getting. Instead allow it to be configured, keeping the current value as default.

Cheers, Allan

-- Allan McRae Arch Linux Developer

More information about the Python-Dev mailing list