Issue 2375: PYTHON3PATH environment variable to supersede PYTHONPATH for multi-Python environments (original) (raw)

Issue2375

Created on 2008-03-18 00:34 by glyph, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (17)
msg63820 - (view)	Author: Glyph Lefkowitz (glyph)	Date: 2008-03-18 00:34
Currently if you have both Python 3 and Python 2 installed, there's no way to indicate that ".py" files in one area are Python 2 syntax and in another area are Python 3 syntax. Being able to distinguish between these would be nice for heterogeneous deployment environments, but is particularly important for developers who are working on Python 3 conversions. It would be good to have a PYTHON3PATH which, if present, would be honored instead of PYTHONPATH. PYTHONPATH could still be honored if PYTHON3PATH was not present, so this is purely a new feature and shouldn't have much impact on compatibility.
msg63841 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2008-03-18 01:46
-1 Sites that need this functionality can emulate this feature in site.py by adding sys.path[1:1] = os.getenv("PYTHON3PATH", "").split(os.pathsep) in py3k installation. I could not find any discussion beyond the original post at http://mail.python.org/pipermail/python-3000/2008-February/012008.html
msg64046 - (view)	Author: Glyph Lefkowitz (glyph)	Date: 2008-03-19 07:38
I don't understand your objection. It sounds like you're objecting, but then suggesting an implementation? The line you suggest might just as easily be present in py3k's default site.py and it would work fine. As I understand it, "sites" don't need this feature, developers do. Let's say I'm working on a typical system with stock python2 and python3 packages installed. Maybe it's a multi-user system, I don't have control over site.py. Normally what I'd do to manipulate sys.path dynamically between different versions would be to install a sitecustomize.py on my one PYTHONPATH entry that detected the version and reacted appropriately. However, writing a program that is both valid python 2 and python 3 is, as I understand it, both tricky and unsupported. Will this specific case be supported so that developers don't need to have root and modify the system python in order to do 2->3 migrations?
msg64057 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2008-03-19 14:47
> I don't understand your objection. > It sounds like you're objecting, but > then suggesting an implementation? This sounds like a kludge addressing a transient problem. It also feels like a solution catering to developers at the expense of user confusion: 1. Why is there PYTHON3PATH, but not PYTHON2PATH? 2. Is PYTHONPATH effectively PYTHON2PATH or PYTHON1PATH? 3. Will PYTHON3PATH be supported forever or deprecated once 2.8 and 3.2 are close enough to support a large common code base? There are many ways to address multi-python site issues. PYTHONPATH is too indiscriminate for some uses. When more fine-grained differentiation between 3.x and 2.x files is needed, a better approach would be a custom loader in metapath that can look for version specific file extensions. I would suggest to wait until there are enough reports from the field on different solutions in actual use before selecting one to standardize on. [I have to mention that perl 5 uses PERL5PATH, but I am not sure whether it is an argument for or against PYTHON3PATH. Can anyone report on pros and cons of PERL5PATH from actual experience?]
msg64095 - (view)	Author: Glyph Lefkowitz (glyph)	Date: 2008-03-19 20:32
The idea is that PYTHON3PATH will be honored in preference to PYTHONPATH, but PYTHONPATH will still be honored. It's exposed only to people who specifically need it. However, I think it's misleading to call the Python 3 transition a "transient" problem. If we're lucky, it's going to be a decade of slow progress. If we're not, it will never end. Try, for example, compiling your system C library without "gets", and see what happens. (And that's been an issue for more than 10 years!) The biggest way to shorten this problem is to provide tools and idioms for developers to use when porting. Specifically like this feature. As far as "version specific file extensions" - I'd be very happy with that but Guido has said several times that he doesn't like it. Feel free to discuss that with him, but this change is far less invasive. Users who don't need it will simply never know about it.
msg64448 - (view)	Author: Ralf Schmitt (schmir)	Date: 2008-03-24 22:48
I once even had the need for a PYTHON24PATH or PYTHON25PATH. (And I think this would be the right approach, i.e. even using the minor version number). However, now I am a happy user of virtualenv, and I do not even have to set PYTHONPATH anymore...
msg64449 - (view)	Author: Ralf Schmitt (schmir)	Date: 2008-03-24 22:49
c modules cannot be used when the minor version number changes. If I remember correctly c modules where the problem for me, when I ran python 2.4 and 2.5 in parallel.
msg64589 - (view)	Author: Georg Brandl (georg.brandl) *	Date: 2008-03-27 13:17
I even run modules compiled for Python 2.2 successfully on 2.5...
msg64591 - (view)	Author: Alexander Belopolsky (belopolsky) *	Date: 2008-03-27 14:05
I have never had a problem of different python versions coexisting on the same machine, but having 32-bit and 64-bit python coexist is much harder. Particularly when 32-bit python is compiled on the 64-bit OS (using -m32 flag). There is a related highlighting this problem. See also , , and . Ideally, I would like to see a mechanism that would allow both standard library and user modules to share machine independent (*.py{,c,o}) files while having separate locations for 32-bit and 64-bit C modules.
msg70469 - (view)	Author: Benjamin Peterson (benjamin.peterson) *	Date: 2008-07-31 02:04
What's the status of this?
msg97724 - (view)	Author: Marc-Andre Lemburg (lemburg) *	Date: 2010-01-13 15:20
Setting up specific environments for each Python version is outside the scope of Python. This is something the user needs to handle using a virtualenv setup, an env-setup shell script or similar approach.
msg97726 - (view)	Author: R. David Murray (r.david.murray) *	Date: 2010-01-13 17:15
I disagree with the closing of this bug on the following grounds: currently, and for the foreseeable future, there will be two python commands on many systems, 'python' and 'python3'. This is unlike the superficially similar situation with different versions of the 2 line. I think this issue should be discussed on python-dev before being closed, and will post a pointer there.
msg97727 - (view)	Author: Marc-Andre Lemburg (lemburg) *	Date: 2010-01-13 17:30
R. David Murray wrote: > > R. David Murray <rdmurray@bitdance.com> added the comment: > > I disagree with the closing of this bug on the following grounds: currently, and for the foreseeable future, there will be two python commands on many systems, 'python' and 'python3'. This is unlike the superficially similar situation with different versions of the 2 line. > > I think this issue should be discussed on python-dev before being closed, and will post a pointer there. You can easily have python3 implemented as shell script setting up PYTHONPATH to whatever your particular Python installation uses. Python itself does not install as python3, so this is not a configuration supported by Python anyway. Also note that it's easy to write a sitecustomize.py which takes care of whatever platform specific side-by-side installation you want to support with an OS or distribution.
msg97728 - (view)	Author: R. David Murray (r.david.murray) *	Date: 2010-01-13 17:46
Yes, it does: rdmurray@maestro:~/python/py3k>ls -l ../ptest/p3/bin total 7328 -rwxr-xr-x 1 rdmurray rdmurray 131 Dec 20 12:22 2to3 -rwxr-xr-x 1 rdmurray rdmurray 119 Dec 20 12:22 idle3 -rwxr-xr-x 1 rdmurray rdmurray 104 Dec 20 12:22 pydoc3 -rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3 lrwxrwxrwx 1 rdmurray rdmurray 16 Jan 13 12:44 python3-config -> python3.2-config -rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3.2 -rwxr-xr-x 1 rdmurray rdmurray 1421 Jan 13 12:44 python3.2-config That's a straight 'make install' from my svn checkout of the py3k trunk.
msg97729 - (view)	Author: Ralf Schmitt (schmir)	Date: 2010-01-13 17:48
> Marc-Andre Lemburg <mal@egenix.com> added the comment: > > > You can easily have python3 implemented as shell script setting up > PYTHONPATH to whatever your particular Python installation uses. yes, and you need to write a python shell script, which resets that variable in case you're starting a python 2 script from your python3 shell script.
msg97730 - (view)	Author: Marc-Andre Lemburg (lemburg) *	Date: 2010-01-13 18:18
R. David Murray wrote: > > R. David Murray <rdmurray@bitdance.com> added the comment: > > Yes, it does: > > rdmurray@maestro:~/python/py3k>ls -l ../ptest/p3/bin total 7328 > -rwxr-xr-x 1 rdmurray rdmurray 131 Dec 20 12:22 2to3 > -rwxr-xr-x 1 rdmurray rdmurray 119 Dec 20 12:22 idle3 > -rwxr-xr-x 1 rdmurray rdmurray 104 Dec 20 12:22 pydoc3 > -rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3 > lrwxrwxrwx 1 rdmurray rdmurray 16 Jan 13 12:44 python3-config -> python3.2-config > -rwxr-xr-x 2 rdmurray rdmurray 3736249 Jan 13 12:43 python3.2 > -rwxr-xr-x 1 rdmurray rdmurray 1421 Jan 13 12:44 python3.2-config > > That's a straight 'make install' from my svn checkout of the py3k trunk. Sorry, forgot about that change. The name of the binaries is changed during installation - after running 'make' you get a 'python' binary. I wonder whether anyone has ever tried building Python on AIX. Renaming binaries after linking is a bad idea on AIX, since the linker then sometimes fails to resolve symbols. I'm still -1 on the change. People setting up PYTHONPATH will know what they are doing and also know which Python version they are going to use with the customized setting and after the transition from Python 2 to 3 we'd be stuck with user environment setups using PYTHON3PATH instead of PYTHONPATH, so removing PYTHON3PATH again won't be easily possible. Besides, if you change PYTHONPATH, you'd also have to change PYTHONHOME, PYTHONSTARTUP, and all the other flags to use the 'PYTHON3' prefix.
msg97782 - (view)	Author: Marc-Andre Lemburg (lemburg) *	Date: 2010-01-14 19:03
Guido pronounced on this on python-dev, so closing the request again.