| msg116467 - (view) |
Author: Sébastien Sablé (sable) |
Date: 2010-09-15 16:30 |
| On AIX, the test test_communicate_pipe_buf in test_subprocess will hang forever (in py3k and py27): test_communicate_pipe_buf (__main__.ProcessTestCase) ... File "Lib/test/test_subprocess.py", line 386, in test_communicate_pipe_buf (stdout, stderr) = p.communicate(string_to_write) File "/san_cis/home/cis/data/bamboo-home-agent-runtime/xml-data/build-dir/RTAIX30-SUP/Python-2.7-svn/Lib/subprocess.py", line 740, in communicate return self._communicate(input) File "/san_cis/home/cis/data/bamboo-home-agent-runtime/xml-data/build-dir/RTAIX30-SUP/Python-2.7-svn/Lib/subprocess.py", line 1257, in _communicate stdout, stderr = self._communicate_with_poll(input) File "/san_cis/home/cis/data/bamboo-home-agent-runtime/xml-data/build-dir/RTAIX30-SUP/Python-2.7-svn/Lib/subprocess.py", line 1320, in _communicate_with_poll input_offset += os.write(fd, chunk) KeyboardInterrupt The comment in this test indicates: # communicate() with writes larger than pipe_buf # This test will probably deadlock rather than fail, if # communicate() does not work properly. So I guess it means communicate() does not work properly on AIX. |
|
|
| msg116932 - (view) |
Author: Sébastien Sablé (sable) |
Date: 2010-09-20 13:35 |
| The problem does not happen with Python 2.6. The difference is that: * in Python 2.6, the subprocess module would try to write at most 512 bytes cf L1221 in subprocess.py chunk = input[input_offset : input_offset + 512] * in Python 2.7 and py3k, the subprocess module will try to write at most _PIPE_BUF bytes cf L1319 in subprocess.py chunk = input[input_offset : input_offset + _PIPE_BUF] When forcing PIPE_BUF to 512 in selectmodule.c, the test will pass. PIPE_BUF seems to be broken on AIX. |
|
|
| msg116952 - (view) |
Author: Sébastien Sablé (sable) |
Date: 2010-09-20 16:21 |
| PIPE_BUF in unistd.h is defined to 32768. The test works with PIPE_BUF changed up to 6144 but won't work with 7168. We could probably use 4096 as a safe value for faster result if needed. I just do not define the value and leave the default of 512 be used. Here is my patch which has been verified to work OK and should be applied to py3k and py27. |
|
|
| msg117031 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *  |
Date: 2010-09-21 06:33 |
| Removing select.PIPE_BUF does not seem a good idea to me because this breaks compatibility. I suggest to simply set it to 512 on AIX. (An ideal solution would be to really determine the actual buffer size in ./configure; this is probably overkill) |
|
|
| msg117040 - (view) |
Author: Sébastien Sablé (sable) |
Date: 2010-09-21 09:05 |
| OK for me. Here is a new patch that defines PIPE_BUF to 512 instead of removing select.PIPE_BUF. |
|
|
| msg118849 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2010-10-16 00:49 |
| Committed to py3k in r85554, 2.7 in r85556. Sébastien, from what you say it sounds like this does not apply to 3.1, so I blocked it there. If this is incorrect let me know and I'll backport it. |
|
|
| msg118998 - (view) |
Author: Sébastien Sablé (sable) |
Date: 2010-10-18 09:24 |
| Thanks R. David. I checked in 3.1 and PIPE_BUF is not defined in the select module, so the default value of 512 is used in subprocess. So no correction is needed for that version. |
|
|