| msg125020 - (view) |
Author: kai zhu (kaizhu) |
Date: 2011-01-02 00:07 |
| i have 2 debian i386 unstable distros. 1) python3.2 (latest hg) running under vps @ linode.com seems ok 2) python3.2 (latest hg) running under colinux (in windows xp) breaks *NOTE 3) python3.2 (release b2) works fine under colinux public@colinux: python3.2 Python 3.2b2+ (py3k, Jan 1 2011, 17:42:23) [GCC 4.4.5] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import subprocess >>> subprocess.Popen('ls') Traceback (most recent call last): File "", line 1, in File "/home/public/i486-pc-linux-gnu/lib/python3.2/subprocess.py", line 708, in __init__ restore_signals, start_new_session) File "/home/public/i486-pc-linux-gnu/lib/python3.2/subprocess.py", line 1136, in _execute_child errpipe_read, errpipe_write = _create_pipe() OSError: [Errno 38] Function not implemented >>> |
|
|
| msg125028 - (view) |
Author: Gregory P. Smith (gregory.p.smith) *  |
Date: 2011-01-02 02:58 |
| The only thing that could cause this error is if the Modules/_posixsubprocess.c subprocess_cloexec_pipe function fails either in the pipe2() call or if HAVE_PIPE2 is not defined one of the fcntl() calls. Can you look to see if HAVE_PIPE2 is defined? I suspect it is but the colinux/windows environment should apperently not define it. configure.in magic will be needed to make sure it does not get defined there. Is your bugreport accurate? This function was included in 3.2b2 so the failure should be the same in both versions. As far as I can see nothing else has changed that should impact that. Did you rerun configure properly on your colinux install? |
|
|
| msg125036 - (view) |
Author: kai zhu (kaizhu) |
Date: 2011-01-02 08:59 |
| tested w/ following debug code. looks like greg is correct - HAVE_PIPE2 should NOT b defined under colinux. diff -r 6fa1e3b94d8f Modules/_posixsubprocess.c --- a/Modules/_posixsubprocess.c Sat Jan 01 22🔞46 2011 +0100 +++ b/Modules/_posixsubprocess.c Sun Jan 02 03:48:47 2011 -0500 @@ -412,10 +412,12 @@ int fds[2]; int res; #ifdef HAVE_PIPE2 + PyErr_Format(PyExc_RuntimeError, "HAVE_PIPE2 = %i, O_CLOEXEC = %i", HAVE_PIPE2, O_CLOEXEC); return NULL; Py_BEGIN_ALLOW_THREADS res = pipe2(fds, O_CLOEXEC); Py_END_ALLOW_THREADS #else + PyErr_Format(PyExc_RuntimeError, "HAVE_PIPE2 not defined, O_CLOEXEC = %i", O_CLOEXEC); return NULL; /* We hold the GIL which offers some protection from other code calling * fork() before the CLOEXEC flags have been set but we can't guarantee * anything without pipe2(). */ b2 release: ./python -c 'import _posixsubprocess; _posixsubprocess.cloexec_pipe()' Traceback (most recent call last): File "", line 1, in RuntimeError: HAVE_PIPE2 not defined, O_CLOEXEC = 524288 latest hg: ./python -c 'import _posixsubprocess; _posixsubprocess.cloexec_pipe()' Traceback (most recent call last): File "", line 1, in RuntimeError: HAVE_PIPE2 = 1, O_CLOEXEC = 524288 |
|
|
| msg125037 - (view) |
Author: kai zhu (kaizhu) |
Date: 2011-01-02 09:04 |
| i used the same almost vanilla configure for both: $ ./configure --prefix=$USERPATH; make |
|
|
| msg125039 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2011-01-02 09:56 |
| Can you please run the script under strace, and report what system call is not implemented? I.e. put import subprocess subprocess.Popen('ls') into a file (foo.py), then run strace -o trace.txt python foo.py Please attach the output in case you cannot identify the problem. |
|
|
| msg125048 - (view) |
Author: kai zhu (kaizhu) |
Date: 2011-01-02 13:31 |
| the culprit was my colinux kernel (2.6.26.8-co-0.7.7.1) did not have pipe2 support (which libc erronenously assumed). updating the kernel fixed the problem. the libc issue is partially discussed @ http://www.0x61.com/forum/linux-kernel-f109/popen2-popen-call-t1229012.html. according to manpage pipe2 was not added to the kernel til 2.6.27 my guess is this affects all unstable debian systems which are running kernels 2.6.26 or older (u decide whether or not using old kernels for debian unstable is an edge case ;) |
|
|
| msg125051 - (view) |
Author: kai zhu (kaizhu) |
Date: 2011-01-02 14:08 |
| hi martin, did an strace & the 'not implemented' system call was pipe2() pipe2 exists in libc (checked w/ ctypes), but is broken for old linux kernels as mentioned previously. |
|
|
| msg125052 - (view) |
Author: Georg Brandl (georg.brandl) * |
Date: 2011-01-02 14:13 |
| Does not seem to be a Python problem then. Thanks for diagnosing! |
|
|
| msg125089 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2011-01-02 20:13 |
| If more people report this, there is still something Python could do: - the configure test could verify that the running kernel actually implements the system call, and undefine HAVE_PIPE2 if that's not the case. Of course this would only help if the resulting binary only ever runs on the same system, and if the kernel is only ever upgraded. - the test could be deferred to run-time, having subprocess_cloexec_pipe fall back to the pipe()/fcntl() branch if the system call fails with ENOSYS. |
|
|
| msg125092 - (view) |
Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2011-01-02 20:17 |
| This bug has brought up a broader issue. the pipe2 syscall on Linux is very new. It is perfectly reasonable to expect a Python binary will be compiled against a C library that has a pipe2() function but run on a system with an older (pre 2.6.27) linux kernel. I have many systems like that at work. I'm going to make subprocess_cloexec_pipe check for a not implemented error from the pipe2 call and use the fallback pipe+fcntl code in that case instead. |
|
|
| msg125093 - (view) |
Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2011-01-02 20:17 |
| This bug has brought up a broader issue. the pipe2 syscall on Linux is very new. It is perfectly reasonable to expect a Python binary will be compiled against a C library that has a pipe2() function but run on a system with an older (pre 2.6.27) linux kernel. I have many systems like that at work. I'm going to make subprocess_cloexec_pipe check for a not implemented error from the pipe2 call and use the fallback pipe+fcntl code in that case instead. |
|
|
| msg125097 - (view) |
Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2011-01-02 20:54 |
| fixed r87651. kaizhu if you still have the ability to run your colinux install with an old 2.6.26 kernel i'd appreciate it if you could explicitly test that the change does the right thing. |
|
|
| msg125110 - (view) |
Author: kai zhu (kaizhu) |
Date: 2011-01-02 21:54 |
| re-tested under old 2.6.26 kernel using previous foo.py example: 1. unpatched python3.2 broken as expected 2. patched python3.2 now works :) strace confirms ENOSYS being raised from pipe2() in both cases |
|
|