Issue 32345: EIO from write() is only fatal if print() contains a newline (original) (raw)

Issue32345

Created on 2017-12-16 09:28 by Creideiki, last changed 2022-04-11 14:58 by admin.

Messages (5)
msg308468 - (view)	Author: (Creideiki)	Date: 2017-12-16 09:28
This is Python 2.7.14 on Gentoo Linux. I ran into an issue where a program crashes if I run it from a terminal, put it in the background, and then close the terminal too soon. Upstream bug report: https://github.com/micahflee/torbrowser-launcher/issues/298 It seems the cause is that a print() call without a newline ignores the EIO returned by the write() syscall, but if there is a newline in the string, that EIO is suddenly a fatal error. Reproducer: $ cat fatal.py #!/bin/env python2.7 import time time.sleep(5) print('A') print('B\nC') print('D') Run this program in the background under strace to see what it does, and while it is sleeping, close its controlling terminal: $ strace -s 256 -o fatal.log -f ./fatal.py & [1] 17974 ^d Now look at the strace log: $ grep write fatal.log 17978 write(1, "A\n", 2) = -1 EIO (Input/output error) 17978 write(1, "B\n", 2) = -1 EIO (Input/output error) 17978 write(2, "Traceback (most recent call last):\n", 35) = -1 EIO (Input/output error) 17978 write(2, " File \"./fatal.py\", line 5, in \n", 41) = -1 EIO (Input/output error) 17978 write(2, " ", 4) = -1 EIO (Input/output error) 17978 write(2, "print('B\\nC')\n", 14) = -1 EIO (Input/output error) 17978 write(2, "IOError", 7) = -1 EIO (Input/output error) 17978 write(2, ": ", 2) = -1 EIO (Input/output error) 17978 write(2, "[Errno 5] Input/output error", 28) = -1 EIO (Input/output error) 17978 write(2, "\n", 1) = -1 EIO (Input/output error) The first print('A') ran and had an EIO error, which was ignored. The second print('B\nC') tried to write 'B', had an EIO error, and crashed. The third print('D') was never attempted.
msg308690 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2017-12-19 23:19
Python 2 doesn't call write(), it calls fwrite() and friends (i.e. it uses the libc's buffered I/O API). Also we don't do anything special if the printed string has a newline in it. So my guess is that it's a bug in the libc.
msg308718 - (view)	Author: (Creideiki)	Date: 2017-12-20 11:09
Hmm. Yes and no; you seem to be correct in that the problem is on the libc level, but Python definitely does something special. By the way, this is glibc 2.26. I tried this C program: #include <stdio.h> #include <unistd.h> int main() { usleep(5 * 1000 * 1000); fwrite("A", 1, 1, stdout); perror("A"); fwrite("\n", 1, 1, stdout); perror("newline 1"); fwrite("B\nC", 3, 1, stdout); perror("BC"); fwrite("\n", 1, 1, stdout); perror("newline 2"); fwrite("D", 1, 1, stdout); perror("D"); fwrite("\n", 1, 1, stdout); perror("newline 3"); return 0; } Which behaves similarly when there's a newline in the middle of the string, in that fwrite() returns with an error after writing the newline: write(3, "A: Success\n", 11) = -1 EIO (Input/output error) write(1, "A\n", 2) = -1 EIO (Input/output error) write(3, "newline 1: Input/output error\n", 30) = -1 EIO (Input/output error) write(1, "B\n", 2) = -1 EIO (Input/output error) write(3, "BC: Input/output error\n", 23) = -1 EIO (Input/output error) write(1, "\n", 1) = -1 EIO (Input/output error) write(3, "newline 2: Input/output error\n", 30) = -1 EIO (Input/output error) write(3, "D: Input/output error\n", 22) = -1 EIO (Input/output error) write(1, "D\n", 2) = -1 EIO (Input/output error) write(3, "newline 3: Input/output error\n", 30) = -1 EIO (Input/output error) However, as can be seen from the result of the perror() calls, every fwrite() after the first buffer flush (which happens on newlines, since stdout to a terminal is line buffered) fails with EIO. This should mean that, at the latest, the second Python print() call should fail. But it doesn't: #!/bin/env python2.7 import time time.sleep(5) print('A') print('E') print('B\nC') print('D') write(1, "A\n", 2) = -1 EIO (Input/output error) write(1, "E\n", 2) = -1 EIO (Input/output error) write(1, "B\n", 2) = -1 EIO (Input/output error) write(2, "Traceback (most recent call last):\n", 35) = -1 EIO (Input/output error) write(2, " File \"./fatal.py\", line 6, in \n", 41) = -1 EIO (Input/output error) So Python does make a difference between the implicit newline added to the end of each print() and a newline in the middle of the user-supplied string.
msg308719 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2017-12-20 11:13
I think the difference here is that Python calls ferror() to tell whether an error occurred on the underlying FILE*. Can you adapt your C program to check ferror()?
msg308732 - (view)	Author: (Creideiki)	Date: 2017-12-20 13:10
I ran this program: #include <stdio.h> #include <unistd.h> #include <errno.h> #include <string.h> void print(const char *s, size_t l) { errno = 0; fwrite(s, l, 1, stdout); int saved_errno = errno; fprintf(stderr, "After \"%s\": ferror(): %i, strerror(): %s\n", s, ferror(stdout), strerror(saved_errno)); clearerr(stdout); } int main() { usleep(5 * 1000 * 1000); print("A", 1); print("\n", 1); print("B\nC", 3); print("\n", 1); print("D", 1); print("\n", 1); return 0; } Got this result: write(2, "After \"A\": ferror(): 0, strerror(): Success\n", 44) = -1 EIO (Input/output error) write(1, "A\n", 2) = -1 EIO (Input/output error) write(2, "After \"\n\": ferror(): 1, strerror(): Input/output error\n", 55) = -1 EIO (Input/output error) write(1, "B\n", 2) = -1 EIO (Input/output error) write(2, "After \"B\nC\": ferror(): 1, strerror(): Input/output error\n", 57) = -1 EIO (Input/output error) write(1, "\n", 1) = -1 EIO (Input/output error) write(2, "After \"\n\": ferror(): 1, strerror(): Input/output error\n", 55) = -1 EIO (Input/output error) write(2, "After \"D\": ferror(): 0, strerror(): Success\n", 44) = -1 EIO (Input/output error) write(1, "D\n", 2) = -1 EIO (Input/output error) write(2, "After \"\n\": ferror(): 1, strerror(): Input/output error\n", 55) = -1 EIO (Input/output error) So, fwrite() with a string which does not contain a newline is buffered and ferror() returns 0. fwrite() with a string which does contain a newline, in the middle or at the end, flushes the buffer and makes ferror() return 1. I think this means that after print('A') gets turned into write(1, "A\n", 2), ferror() should still be 1.