Issue 30783: Spawned subprocesses don't respect environment (original) (raw)

The following example shows that we are indeed changing PATH, but the subprocess does not acknowledge it in Windows 7 x64. Also note this works in Linux (Ubuntu 16.04).

import os import subprocess from contextlib import contextmanager from tempfile import TemporaryDirectory

def get_python_path(): return subprocess.check_output( ['python', '-c', 'import sys;print(sys.executable)'] ).decode().strip()

@contextmanager def temp_chdir(cwd=None): with TemporaryDirectory() as d: origin = cwd or os.getcwd() os.chdir(d)

    try:
        yield d
    finally:
        os.chdir(origin)

def create_venv(d, pypath=None): command = ['virtualenv', d] if pypath: command.extend(['-p', pypath]) subprocess.call(command)

@contextmanager def venv(d): if os.path.exists(os.path.join(d, 'bin')): # no cov venv_exe_dir = os.path.join(d, 'bin') elif os.path.exists(os.path.join(d, 'Scripts')): venv_exe_dir = os.path.join(d, 'Scripts') else: raise OSError('Unable to locate executables directory.')

old_path = os.environ['PATH']
os.environ['PATH'] = '{}{}{}'.format(venv_exe_dir, os.pathsep, old_path)
yield
os.environ['PATH'] = old_path

def test_venv(): with temp_chdir() as d: d = os.path.join(d, 'test_env') create_venv(d) global_python = get_python_path() print('PATH', os.environ['PATH'][:140])

    with venv(d):
        print('PATH', os.environ['PATH'][:140])
        venv_python = get_python_path()

    assert global_python != venv_python
    assert global_python == get_python_path()

subprocess.Popen calls CreateProcess on Windows, which searches for an unqualified executable in the command line as follows:

1. The directory from which the application loaded.
2. The current directory for the parent process. (Starting with
   Vista, the current directory is excluded from this search if
   the environment variable NoDefaultCurrentDirectoryInExePath is
   defined.)
3. The 32-bit Windows system directory. Use the GetSystemDirectory
   function to get the path of this directory.
4. The 16-bit Windows system directory. There is no function that
   obtains the path of this directory, but it is searched. The
   name of this directory is System.
5. The Windows directory. Use the GetWindowsDirectory function to
   get the path of this directory.
6. The directories that are listed in the PATH environment
   variable.

Thus searching for "python" will always find "python.exe" from the application directory.

To work around this, you can use shutil.which() to find python.exe on PATH and pass it as the executable argument. For example:

os.environ['PATH'] = ';'.join([r'C:\Program Files\Python35', old_path])
python35 = shutil.which('python')

>>> print(python35)
C:\Program Files\Python35\python.EXE

>>> _ = subprocess.call('python -V')
Python 3.6.1

>>> _ = subprocess.call('python -V', executable=python35)
Python 3.5.2

cmd.exe implements its own search, like shutil.which, and uses the CreateProcess lpApplicationName parameter that corresponds to the Popen executable parameter. But in general (not always) it's better to use shutil.which because you don't have to worry about the security problems that come with using the shell.