Issue 11191: test_search_cpp error on AIX (with xlc) (original) (raw)

Created on 2011-02-11 15:02 by sable, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 5206 closed Michael.Felt,2018-01-16 18:49
PR 8709 merged Michael.Felt,2018-08-08 22:52
Messages (10)
msg128399 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-11 15:02
I have the following error in distutils on AIX. ====================================================================== ERROR: test_search_cpp (distutils.tests.test_config_cmd.ConfigTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/unixccompiler.py", line 166, in preprocess self.spawn(pp_args) File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/ccompiler.py", line 911, in spawn spawn(cmd, dry_run=self.dry_run) File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/spawn.py", line 34, in spawn _spawn_posix(cmd, search_path, dry_run=dry_run) File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/spawn.py", line 138, in _spawn_posix % (cmd[0], exit_status)) distutils.errors.DistutilsExecError: command 'xlc' failed with exit status 40 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/tests/test_config_cmd.py", line 47, in test_search_cpp match = cmd.search_cpp(pattern='xxx', body='// xxx') File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/command/config.py", line 203, in search_cpp src, out = self._preprocess(body, headers, include_dirs, lang) File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/command/config.py", line 126, in _preprocess self.compiler.preprocess(src, out, include_dirs=include_dirs) File "/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/distutils/unixccompiler.py", line 168, in preprocess raise CompileError(msg) distutils.errors.CompileError: command 'xlc' failed with exit status 40 I haven't investigated yet.
msg220898 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-17 21:32
After three years can we close this test failure as "out of date"?
msg220935 - (view) Author: Sébastien Sablé (sable) Date: 2014-06-18 12:37
I don't have any AIX environment to try to reproduce the issue, so feel free to close it.
msg309982 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-01-15 13:59
After even more years - I see the same test failing, just a bit different. And, others.... in short: ./python Lib/test/test_distutils.py ... Ran 245 tests in 10.337s FAILED (errors=7, skipped=31) I have managed to get this to: FAILED (errors=1, skipped=37) And this is still: ERROR: test_search_cpp (distutils.tests.test_config_cmd.ConfigTestCase) So, moving forward now.
msg309999 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-01-15 16:53
a) Is this normal? root@x065:[/data/prj/python/git/python3-3.7]./python -m unittest test.test_distutils ---------------------------------------------------------------------- Ran 0 tests in 0.000s OK In short - at the start I had: ./python Lib/test/test_distutils.py ... ERROR: test_run (distutils.tests.test_build_clib.BuildCLibTestCase) ERROR: test_build_ext (distutils.tests.test_build_ext.BuildExtTestCase) ERROR: test_get_outputs (distutils.tests.test_build_ext.BuildExtTestCase) ERROR: test_build_ext (distutils.tests.test_build_ext.ParallelBuildExtTestCase) ERROR: test_get_outputs (distutils.tests.test_build_ext.ParallelBuildExtTestCase) ERROR: test_search_cpp (distutils.tests.test_config_cmd.ConfigTestCase) ERROR: test_record_extensions (distutils.tests.test_install.InstallTestCase) That is now down to: ERROR: test_search_cpp (distutils.tests.test_config_cmd.ConfigTestCase) There were two issues for search_cpp: a) xlc -E needs -C to include comments in the output - this should be fixed b) xlc -E does not accept -o as an argument - output must be to stdout, and sadly I am not clever enough to fix that. Help appreciated. As is: the error was: ERROR: test_search_cpp (distutils.tests.test_config_cmd.ConfigTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/prj/python/git/xlc-python3-3.7/Lib/distutils/unixccompiler.py", line 107, in preprocess self.spawn(pp_args) File "/data/prj/python/git/xlc-python3-3.7/Lib/distutils/ccompiler.py", line 909, in spawn spawn(cmd, dry_run=self.dry_run) File "/data/prj/python/git/xlc-python3-3.7/Lib/distutils/spawn.py", line 36, in spawn _spawn_posix(cmd, search_path, dry_run=dry_run) File "/data/prj/python/git/xlc-python3-3.7/Lib/distutils/spawn.py", line 159, in _spawn_posix % (cmd, exit_status)) distutils.errors.DistutilsExecError: command 'xlc_r' failed with exit status 40 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/data/prj/python/git/xlc-python3-3.7/Lib/distutils/tests/test_config_cmd.py", line 49, in test_search_cpp match = cmd.search_cpp(pattern='xxx', body='/* xxx */') File "/data/prj/python/git/xlc-python3-3.7/Lib/distutils/command/config.py", line 201, in search_cpp src, out = self._preprocess(body, headers, include_dirs, lang) File "/data/prj/python/git/xlc-python3-3.7/Lib/distutils/command/config.py", line 124, in _preprocess self.compiler.preprocess(src, out, include_dirs=include_dirs) File "/data/prj/python/git/xlc-python3-3.7/Lib/distutils/unixccompiler.py", line 109, in preprocess raise CompileError(msg) distutils.errors.CompileError: command 'xlc_r' failed with exit status 40 and it is now: ====================================================================== ERROR: test_search_cpp (distutils.tests.test_config_cmd.ConfigTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/prj/python/git/python3-3.7/Lib/distutils/tests/test_config_cmd.py", line 49, in test_search_cpp match = cmd.search_cpp(pattern='xxx', body='/* xxx */') File "/data/prj/python/git/python3-3.7/Lib/distutils/command/config.py", line 206, in search_cpp file = open(out) FileNotFoundError: [Errno 2] No such file or directory: '_configtest.i' ********* The code I do not know how to fix is: File: Lib/distutils/command/config.py def _preprocess(self, body, headers, include_dirs, lang): src = self._gen_temp_sourcefile(body, headers, lang) out = "_configtest.i" self.temp_files.extend([src, out]) self.compiler.preprocess(src, out, include_dirs=include_dirs) return (src, out) See https://github.com/aixtools/cpython/tree/bpo-11191 for my changes
msg310518 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-01-23 18:52
Just tested the patch presented. For Python3-3.6.4 the patch in PR-5206 works as is, however, for Python3-3.5.4 - the patch to Lib/test/support/__init__.py does not work. So, for Python3-3.5, Python3-3.6 and Python3-master - the test test_search_cpp is resolved. As that is the "literal" question here - I am removing the patch to Lib/test/support/__init__.py so that the same tests fail (for different reasons). I'll open a new issue for these tests - so that this PR can be applied to all current Python3 branches (and skip the test_search_cpp test when the compiler is not gcc on AIX). HTH!
msg318522 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-06-03 06:05
I hope this can be reviewed and eventually closed - not because it is X years old and unresolved - but because it is resolved for the latest branches! Thx
msg321919 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-07-18 21:00
@tarek - anything specific/extra you need?
msg327047 - (view) Author: Michael Felt (Michael.Felt) * Date: 2018-10-04 10:47
The current PR8709 resolves two issues, not one - so will create a new issue for the second element. Am also shorting the NEWS text, with the expanded explanation here as: * skip the distutils test 'test_search_cpp' when not gcc as compiler because not all compilers, e.g. IBM xlc et al do not support redirected stdout when using the argument -E (aka cpp_preprocessing) while gcc is know to support the pair -E -o outputfile The additional find while researching this issue: * add the argument '-C' for AIX xlc cpp processing so comments are included in the cpp output rather than replaced by a single ' ' (space) char.
msg332658 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2018-12-28 14:03
New changeset ed57e13df60ce28ba89bd49c9c5a15b1d9bf79c7 by Nick Coghlan (Michael Felt) in branch 'master': bpo-11191: skip unsupported test_distutils case for AIX with xlc (GH-8709) https://github.com/python/cpython/commit/ed57e13df60ce28ba89bd49c9c5a15b1d9bf79c7
History
Date User Action Args
2022-04-11 14:57:12 admin set github: 55400
2018-12-28 14:05:28 ncoghlan set status: open -> closedstage: patch review -> resolvedresolution: fixedversions: - Python 3.5, Python 3.6, Python 3.7
2018-12-28 14:03:21 ncoghlan set nosy: + ncoghlanmessages: +
2018-10-04 10:47:55 Michael.Felt set messages: +
2018-08-08 22:52:58 Michael.Felt set pull_requests: + <pull%5Frequest8196>
2018-07-18 21:00:10 Michael.Felt set messages: +
2018-06-03 06:05:34 Michael.Felt set messages: + versions: + Python 3.8
2018-01-23 18:52:13 Michael.Felt set messages: + versions: + Python 3.5, Python 3.6
2018-01-16 18:49:17 Michael.Felt set keywords: + patchstage: patch reviewpull_requests: + <pull%5Frequest5060>
2018-01-15 16:53:23 Michael.Felt set messages: +
2018-01-15 14:09:20 BreamoreBoy set nosy: - BreamoreBoy
2018-01-15 13:59:31 Michael.Felt set versions: + Python 3.7, - Python 3.2
2018-01-15 13:59:15 Michael.Felt set nosy: + Michael.Feltmessages: +
2014-06-18 12:37:08 sable set messages: +
2014-06-17 21:32:36 BreamoreBoy set type: behaviormessages: + nosy: + BreamoreBoy
2011-02-11 15:02:23 sable set nosy:sable, tarek, eric.araujotitle: test_search_cpp error on AIX (aith xlc) -> test_search_cpp error on AIX (with xlc)
2011-02-11 15:02:15 sable create