Issue 20754: Distribution.parse_config_files uses interpolation (original) (raw)

Created on 2014-02-24 00:27 by alunduil, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
disable-distutils-string-interpolation.patch noirbizarre,2016-12-12 15:49 review
Messages (13)
msg212048 - (view) Author: Alex Brandt (alunduil) Date: 2014-02-24 00:27
I first noticed this issue when configuring nose via setup.cfg and reported the behavior in this issue on their tracker: https://github.com/nose-devs/nose/issues/733 I'll repaste the important bits here: When using a setup.cfg with the following contents (including minor variations): [nosetests] logging-format = %(lineno)d: %(process)d: %(message)s With any setup.py file the following error is raised when trying to run setup.py (python2.7 used for example, error also occurs in 3.3): alunduil@elijah margarine % python2.7 setup.py nosetests Traceback (most recent call last): File "setup.py", line 108, in setup(**PARAMS) File "/usr/lib64/python2.7/distutils/core.py", line 125, in setup dist.parse_config_files() File "/usr/lib64/python2.7/distutils/dist.py", line 397, in parse_config_files val = parser.get(section,opt) File "/usr/lib64/python2.7/ConfigParser.py", line 623, in get return self._interpolate(section, option, value, d) File "/usr/lib64/python2.7/ConfigParser.py", line 669, in _interpolate option, section, rawval, e.args[0]) ConfigParser.InterpolationMissingOptionError: Bad value substitution: section: [nosetests] option : logging-format key : pathname rawval : %(pathname)s:%(lineno)d: %(process)d: %(message)s The suggested fix of using %% to pass the interpolation through to nose requires that distutils use SafeConfigParser rather than ConfigParser. I've verified that this does indeed appear to work (at least fixing the distutils interaction). The only place the change needs to be made is on lines 378 and 386 in distutils/dist.py. I can attach a patch or send a pull request with this fix if there are no objections to this solution.
msg277668 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-09-28 22:07
Thanks for the report, Alex. SafeConfigParser has been renamed to ConfigParser in Python 3.2 and distutils already uses ConfigParser in Python 3: From Distribution.parse_config_files(): ... parser = ConfigParser() For 2.7, can you try it with setuptools? Perhaps we should keep 2.7 as-is and let setuptools do its magic.
msg277669 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2016-09-28 22:10
Do the changes in address this concern?
msg283006 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2016-12-12 14:34
Looking more closely now and as a result of the report in , I see that this issue is another separate manifestation of the issue addressed in .
msg283014 - (view) Author: Axel Haustant (noirbizarre) * Date: 2016-12-12 15:49
I just attached the patch submitted on . The fix is to simply give interpolation=None as ConfigParser parameter (as documented in the official Python 3.x ConfigParser documentation)
msg283023 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-12-12 16:37
Thanks! We also need a test case for the new behavior.
msg283068 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2016-12-13 03:17
I forget if there was a reason for choosing RawConfigParser over ConfigParaer with interpolation=None. I'd like to know that before choosing the latter. I'd also like to see if appropriate the implementation patched in Setuptools, providing compatibility for older Pythons. Also, +1 for a test.
msg283194 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2016-12-14 15:06
I reviewed the docs for configparser, and I agree - using ConfigParser(interpolation=None) is preferable to the "legacy" RawConfigParser.
msg283203 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2016-12-14 16:06
I've created a backport implementation in https://github.com/pypa/setuptools/issues/889. All that remains then is to have a test for Python, which can probably borrow from the fix for . I'll leave it to others to draft the patch for the test.
msg299287 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2017-07-27 02:40
While investigating https://github.com/pypa/setuptools/issues/1062, I discovered that the implementation in the attached patch is insufficient. The 'parse_config_files' calls ConfigParser.__init__ in two places, before the for loop and at the end of each loop iteration. The parameters to both calls needs to be the same, or the BasicInterpolation behavior is revived.
msg299291 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2017-07-27 03:20
Ack. The interpolation behavior was also present in Python 2.7. I missed that when I found this ticket and assumed it was similar to issue 20120. In this ticket, the user is requesting that distutils actually change the behavior from performing interpolation to no longer performing it. This change is apparent now that Setuptools has a proper test for the desired behavior.
msg299292 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2017-07-27 03:21
Given that it's a change in behavior and not a bugfix, I don't see how this change could go into anything but the next release.
msg386347 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-02-03 18:21
Distutils is now deprecated (see PEP 632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils. If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools
History
Date User Action Args
2022-04-11 14:57:59 admin set github: 64953
2021-02-03 18:21:22 steve.dower set status: open -> closednosy: + steve.dowermessages: + resolution: out of datestage: patch review -> resolved
2017-07-27 03:21:07 jaraco set messages: + versions: - Python 2.7, Python 3.5, Python 3.6
2017-07-27 03:20:36 jaraco set title: Distribution.parse_config_files uses interpolation since Python 3 -> Distribution.parse_config_files uses interpolationmessages: + versions: + Python 2.7
2017-07-27 02:40:42 jaraco set messages: +
2016-12-14 16:06:05 jaraco set messages: +
2016-12-14 15:06:55 jaraco set messages: +
2016-12-13 03:17:30 jaraco set messages: +
2016-12-12 16:37:40 berker.peksag set messages: + stage: resolved -> patch review
2016-12-12 16:34:26 berker.peksag link issue28935 superseder
2016-12-12 15:49:31 noirbizarre set files: + disable-distutils-string-interpolation.patchnosy: + noirbizarremessages: + keywords: + patch
2016-12-12 14:34:21 jaraco set status: closed -> openversions: + Python 3.5, Python 3.6, Python 3.7, - Python 2.7, Python 3.3, Python 3.4title: distutils should use SafeConfigParser -> Distribution.parse_config_files uses interpolation since Python 3messages: + resolution: out of date -> (no value)
2016-09-28 22:10:40 jaraco set messages: +
2016-09-28 22:07:20 berker.peksag set status: open -> closednosy: + berker.peksag, jaracomessages: + resolution: out of datestage: resolved
2014-02-24 23:38:38 Arfrever set nosy: + tarek, eric.araujoversions: + Python 3.4
2014-02-24 00:27:50 alunduil create