Issue 4887: environment inspection and manipulation API is buggy, inconsistent with "Python philosophy" for wrapping native APIs (original) (raw)
Created on 2009-01-08 22:08 by exarkun, last changed 2022-04-11 14:56 by admin. This issue is now closed.
Messages (6)
Author: Jean-Paul Calderone (exarkun) * 
Date: 2009-01-08 22:08
According to a recent thread on python-dev, Python APIs with the same name as native APIs can be trusted to just pass through to the platform API call (<http://mail.python.org/pipermail/python-dev/2009-January/084899.html>). This isn't the case for the environment APIs in the os module. For example, os.getenv doesn't always return the actual value of a key in the environment. Instead, it looks in the os.environ pseudo-dict to get the answer. This can be wrong because os.environ is a copy of the environment which any non-Python API will use, and as a copy it can get out of sync with the real environment. One way it can get out of sync is via os.environ.pop (at least on POSIX - os.environ has different implementations on different platforms, I find the way an implementation is selected to be rather confusing and I have not tracked down the precise behavior for all platforms):
import os os.environ.pop('PATH') '/home/exarkun/.local/sbin:/home/exarkun/.local/bin:/home/exarkun/Projects/combinator_paths/bincache:/home/exarkun/.local/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games' os.getenv('PATH') os.system("echo $PATH") /home/exarkun/.local/sbin:/home/exarkun/.local/bin:/home/exarkun/Projects/combinator_paths/bincache:/home/exarkun/.local/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games 0
Another way is via calls to the platform environment manipulation APIs (setenv, unsetenv or putenv). These might be invoked by a third-party library which is exposed to Python or via ctypes.
`environ´ itself is a C API, though it's a char**, whereas os.environ is a Python dict. It's probably convenient for os.environ to remain as a dict, since that's so much simpler to manipulate and inspect than the C data structure used. However, it'd be good if were always in sync with the real process environment. A good way to do this would probably be to stop maintaining a copy of the environment and always pass through to one of the platform APIs to satisfy a method call onto it.
Author: Mark Lawrence (BreamoreBoy) *
Date: 2014-06-16 21:09
Is the OP interested in taking this forward?
Author: Jean-Paul Calderone (exarkun) *
Date: 2014-06-17 12:32
What are the chances a future Python 2.x release will include any fix developed for this issue?
Author: Mark Lawrence (BreamoreBoy) *
Date: 2014-06-17 12:39
Good as 2.7 is in support until 2020.
Author: Martin Panter (martin.panter) *
Date: 2014-09-22 02:12
I suspect the $PATH example is a bad example. On my Linux setup, my initial PATH value is a customized path. After popping it and calling system(), I see a simpler default path, which is probably re-initialized somewhere else.
I suspect there is no environ.pop() bug. Everthing works according the documentation if you use another environment variable:
$ TEST_VARIABLE=value python Python 3.4.0 (default, Mar 17 2014, 23:20:09) [GCC 4.8.2 20140206 (prerelease)] on linux Type "help", "copyright", "credits" or "license" for more information.
import os os.system('echo "$TEST_VARIABLE"') value 0 os.environ.pop("TEST_VARIABLE") 'value' os.system('echo "$TEST_VARIABLE"')
0
The other point in this report, about synchronizing “os.environ” with the C APIs, is a duplicate of Issue 1159.
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2014-11-26 13:41
Agreed. environ.pop() was fixed a long time ago with . It seems that all mutable methods of the environ pseudo-dict are now correctly reflected to the platform environ.
The other direction (updates from C code should be reflected in os.environ) is tracked by .
History
Date
User
Action
Args
2022-04-11 14:56:43
admin
set
github: 49137
2014-11-26 13:41:51
amaury.forgeotdarc
set
status: open -> closed
nosy: + amaury.forgeotdarc
messages: +
superseder: os.getenv() not updated after external module uses C putenv()
resolution: duplicate
2014-09-22 02:12:49
martin.panter
set
messages: +
2014-06-17 21:32:36
martin.panter
set
nosy: + martin.panter
2014-06-17 12:39:22
BreamoreBoy
set
messages: +
versions: + Python 2.7, Python 3.5
2014-06-17 12:32:19
exarkun
set
messages: +
2014-06-16 21:09:26
BreamoreBoy
set
nosy: + BreamoreBoy
messages: +
2012-11-05 13:28:59
Ramchandra Apte
set
versions: + Python 3.4, - Python 3.1, Python 2.7, Python 3.2
2010-07-11 09:48:11
BreamoreBoy
set
versions: + Python 3.1, Python 2.7, Python 3.2
2009-09-27 16:42:50
eric.araujo
set
nosy: + eric.araujo
2009-01-08 22:08:06
exarkun
create