| msg315886 - (view) |
Author: Martin Husemann (MartinHusemann) |
Date: 2018-04-29 10:16 |
| When building python extensions in the background w/o stdin (and stderr and stdout redirected to a log file), the invocation of setup.py fails. Normal build in a shell: Example from pyexpat: > /usr/pkg/bin/python3.6 setup.py build running build running build_ext building 'pyexpat' extension creating build creating build/temp.netbsd-8.99.14-amd64-3.6 creating build/temp.netbsd-8.99.14-amd64-3.6/Modules gcc -DNDEBUG -O2 -pipe -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/pkg/include -O2 -pipe -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/pkg/include/python3.6 -I/usr/include -I/usr/pkg/include/python3.6 -fPIC -DHAVE_EXPAT_H -I/work/pkgobj/textproc/py-expat/work/.buildlink/include -I/usr/pkg/include/python3.6 -c Modules/pyexpat.c -o build/temp.netbsd-8.99.14-amd64-3.6/Modules/pyexpat.o creating build/lib.netbsd-8.99.14-amd64-3.6 gcc -pthread -shared -L. -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -L/usr/X11R7/lib -Wl,-R/usr/X11R7/lib -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -L/usr/X11R7/lib -Wl,-R/usr/X11R7/lib -L/usr/lib -Wl,-R/usr/lib -Wl,-R/usr/pkg/lib -O2 -pipe -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/pkg/include/python3.6 -I/usr/include -I/usr/pkg/include/python3.6 build/temp.netbsd-8.99.14-amd64-3.6/Modules/pyexpat.o -L/work/pkgobj/textproc/py-expat/work/.buildlink/lib -L/usr/pkg/lib -Wl,-R/work/pkgobj/textproc/py-expat/work/.buildlink/lib -lexpat -lpython3.6 -o build/lib.netbsd-8.99.14-amd64-3.6/pyexpat.so But if invoked as > ( sleep 15; /usr/pkg/bin/python3.6 setup.py build ) >/tmp/log 2>&1 & > ^D and then watching /tmp/log from another session: Fatal Python error: Py_Initialize: can't initialize sys standard streams OSError: [Errno 9] Bad file descriptor Current thread 0x000075b4e4cee800 (most recent call first): [1] Abort trap (core dumped) /usr/pkg/bin/pyt... |
|
|
| msg316183 - (view) |
Author: Xavier de Gaye (xdegaye) *  |
Date: 2018-05-04 18:24 |
| I cannot reproduce the problem with Python 3.6.5, building pyephem and with the command: ( sleep 15; python setup.py build ) >foo.log 2>&1 |
|
|
| msg317110 - (view) |
Author: Martin Husemann (MartinHusemann) |
Date: 2018-05-19 10:36 |
| You need to exit the parent shell, to get the original stdin revoke(2)'d. That is: the Ctrl-D in the original descritpion is not line noise. Sorry, should have been more explicit (or used "exit" or something instead) |
|
|
| msg317114 - (view) |
Author: Leonardo Taccari (leot) |
Date: 2018-05-19 11:19 |
| A simpler way to reproduce that (probably this problem is not limited to building but to sys module): (sleep 10; python3.6 -c 'import sys' >/tmp/log 2>&1) & exit And in `/tmp/log': Fatal Python error: Py_Initialize: can't initialize sys standard streams OSError: [Errno 9] Bad file descriptor Current thread 0x00007226e0ab3800 (most recent call first): |
|
|
| msg317117 - (view) |
Author: Leonardo Taccari (leot) |
Date: 2018-05-19 13:31 |
| And here the backtrace of the corresponding core (this is on NetBSD/amd64 8.99.15 with lang/python36 package of today pkgsrc-current): % gdb -core python3.6.core `which python3.6` Reading symbols from /usr/pkg/bin/python3.6...done. [New process 1] Core was generated by `python3.6'. Program terminated with signal SIGABRT, Aborted. #0 0x000076255932924a in _lwp_kill () from /usr/lib/libc.so.12 (gdb) bt #0 0x000076255932924a in _lwp_kill () from /usr/lib/libc.so.12 #1 0x0000762559328e65 in abort () at /usr/src/lib/libc/stdlib/abort.c:74 #2 0x000076255a749d33 in Py_FatalError (msg=msg@entry=0x76255a7dea20 "Py_Initialize: can't initialize sys standard streams") at Python/pylifecycle.c:1462 #3 0x000076255a74af4c in _Py_InitializeEx_Private (install_sigs=1, install_importlib=1) at Python/pylifecycle.c:450 #4 0x000076255a764902 in Py_Main (argc=argc@entry=3, argv=argv@entry=0x76255ab01060) at Modules/main.c:700 #5 0x0000000000400e2f in main (argc=3, argv=) at ./Programs/python.c:69 (gdb) f 4 #4 0x000076255a764902 in Py_Main (argc=argc@entry=3, argv=argv@entry=0x76255ab01060) at Modules/main.c:700 700 Py_Initialize(); (gdb) f 3 #3 0x000076255a74af4c in _Py_InitializeEx_Private (install_sigs=1, install_importlib=1) at Python/pylifecycle.c:450 450 Py_FatalError( (gdb) list 445 if (_PyTraceMalloc_Init() < 0) 446 Py_FatalError("Py_Initialize: can't initialize tracemalloc"); 447 448 initmain(interp); /* Module __main__ */ 449 if (initstdio() < 0) 450 Py_FatalError( 451 "Py_Initialize: can't initialize sys standard streams"); 452 453 /* Initialize warnings. */ 454 if (PySys_HasWarnOptions()) { Please let me know if any further information is needed and I will try to help! Thank you! |
|
|
| msg317121 - (view) |
Author: Leonardo Taccari (leot) |
Date: 2018-05-19 14:45 |
| After testing is_valid_fd() (from Python/pylifecycle.c) separately (an `is_valid_fd.c' file will be attached to ease reproduction) I think that also NetBSD is affected by bpo-30225. Using dup(2) (at is currently done in NetBSD): % cc -o ivf is_valid_fd.c % sleep 5 && ./ivf > /tmp/log & exit [... the terminal is closed via ^D ...] % cat /tmp/log 0: 1 1: 1 2: 1 Using fstat(2): % cc -DUSE_FSTAT -o ivf is_valid_fd.c % sleep 5 && ./ivf > /tmp/log & exit [... the terminal is closed via ^D ...] % cat /tmp/log 0: 0 1: 1 2: 0 The possible attached patch seems to fix the problem reported by Martin. |
|
|
| msg355179 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2019-10-22 23:30 |
| I'm quite that this issue has been fixed by bpo-30225 which is part Python 3.7.4, Python 3.8.0 and newer. I mark the issue as duplicate of bpo-30225. Please reopen the bug if these versions still fail on NetBSD. commit 1c4670ea0cc3d208121af11b9b973e6bb268e570 Author: Victor Stinner <victor.stinner@gmail.com> Date: Thu May 4 00:45:56 2017 +0200 bpo-30225: Fix is_valid_fd() on macOS Tiger (#1443) is_valid_fd() now uses fstat() instead of dup() on macOS to return 0 on a pipe when the other side of the pipe is closed. fstat() fails with EBADF in that case, whereas dup() succeed. |
|
|
| msg355321 - (view) |
Author: Leonardo Taccari (leot) |
Date: 2019-10-24 10:38 |
| Hello Victor, I can confirm that I can no longer reproduce this problem with Python 3.7.5 on NetBSD and according a double-check to Python/pylifecycle.c:is_valid_fd() now fstat() is always used on NetBSD. Thanks for the update and for looking at it! |
|
|
| msg355323 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-10-24 10:39 |
| > I can confirm that I can no longer reproduce this problem with Python 3.7.5 on NetBSD Great! |
|
|