Issue 16597: file descriptor not being closed with context manager on IOError when device is full (original) (raw)

process

Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, asvetlov, benjamin.peterson, christian.heimes, hynek, jcea, pitrou, python-dev, serhiy.storchaka, stutzbach, udoprog
Priority: normal Keywords: patch

Created on 2012-12-02 23:24 by udoprog, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
bufferedio_finally_close.patch serhiy.storchaka,2012-12-03 12:28 review
bufferedio_finally_close_2.patch serhiy.storchaka,2012-12-20 17:37 review
bufferedio_finally_close-3.2_2.patch serhiy.storchaka,2012-12-20 18:14 Patch for 3.2 review
bufferedio_finally_close-2.7_2.patch serhiy.storchaka,2012-12-20 18:18 Patch for 2.7 review
Messages (17)
msg176816 - (view) Author: John-John Tedro (udoprog) Date: 2012-12-02 23:24
In 3.2.2 and 3.2.3 on linux64, when running the following code. try: print("Writing to /dev/full") with open("/dev/full", "w") as f: f.write("Write to full device") except: print("Catch, file closed?") Using strace -e close ~/usr/python3.2.3/bin/python3 test.py ... Writing to /dev/full Catch, file closed? close(3) = 0 The file descriptor being used (3) to attempt writing to /dev/full is not closed until the process exits. I expected this to be closed when leaving the context manager.
msg176818 - (view) Author: John-John Tedro (udoprog) Date: 2012-12-02 23:31
Originally discovered on http://stackoverflow.com/questions/13649330/what-happens-to-file-descriptors-in-python-3-when-close-fails
msg176830 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-03 09:22
>>> f = open("/dev/full", "wb", buffering=0) >>> f.write(b"Write to full device") Traceback (most recent call last): File "", line 1, in OSError: [Errno 28] No space left on device >>> f.close() >>> f.closed True >>> f = open("/dev/full", "wb") >>> f.write(b"Write to full device") 20 >>> f.close() Traceback (most recent call last): File "", line 1, in OSError: [Errno 28] No space left on device >>> f.closed False Python 2 has the same behavior using io.open.
msg176831 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-03 09:24
The bug only in C implementation. >>> import _pyio >>> f = _pyio.open("/dev/full", "wb") >>> f.write(b"Write to full device") 20 >>> f.close() Traceback (most recent call last): File "", line 1, in File "/home/serhiy/py/cpython/Lib/_pyio.py", line 732, in close self.flush() File "/home/serhiy/py/cpython/Lib/_pyio.py", line 1121, in flush self._flush_unlocked() File "/home/serhiy/py/cpython/Lib/_pyio.py", line 1128, in _flush_unlocked n = self.raw.write(self._write_buf) OSError: [Errno 28] No space left on device >>> f.closed True
msg176839 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-03 12:25
Here is a patch which calls close() on underlying stream even if flush() raises an exception. I am not sure that I correctly set a context exception. There is no other examples in the code.
msg177832 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-20 16:15
There are a lot of people in the nosy list already. Does anyone have enough experience with exception machinery to review the patch (especially C part)?
msg177838 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-20 17:37
Patch updated with Benjamin's nit.
msg177840 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-12-20 17:55
New changeset b0602a1eb3f6 by Benjamin Peterson in branch '3.3': call close on the underlying stream even if flush raises (closes #16597) http://hg.python.org/cpython/rev/b0602a1eb3f6 New changeset 142012e47c3b by Benjamin Peterson in branch 'default': merge 3.3 (#16597) http://hg.python.org/cpython/rev/142012e47c3b
msg177842 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-20 18:14
Here is a patch for 3.2.
msg177843 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-20 18:18
Here is a patch for 2.7.
msg177846 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-12-20 18:23
I think we can leave 3.2 alone.
msg177847 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-12-20 18:24
New changeset b6ff6ac1f049 by Benjamin Peterson in branch '2.7': call close on the underlying stream even if flush raises (#16597) http://hg.python.org/cpython/rev/b6ff6ac1f049
msg177848 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-20 18:26
What is the peculiarity of 3.2?
msg177851 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-20 18:52
About Misc/NEWS. Actually fixed close() methods of: Python implementation of BaseIO (C implementation already do right things), C implemettation of Buffered(Reader|Writer Random) (Python implementation already do right things) and both implementations of TextIOWrapper.
msg177852 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-12-20 18:56
New changeset 54372f38932e by Benjamin Peterson in branch '3.3': improve message (#16597) http://hg.python.org/cpython/rev/54372f38932e New changeset faaac6ceffff by Benjamin Peterson in branch 'default': merge 3.3 (#16597) http://hg.python.org/cpython/rev/faaac6ceffff New changeset a057d9985fb8 by Benjamin Peterson in branch '2.7': add news note (#16597) http://hg.python.org/cpython/rev/a057d9985fb8
msg177853 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-20 19:10
Thank you for all commits, Benjamin. What is the peculiarity of 3.2?
msg177854 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-12-20 19:11
2012/12/20 Serhiy Storchaka <report@bugs.python.org>: > > Serhiy Storchaka added the comment: > > Thank you for all commits, Benjamin. > > What is the peculiarity of 3.2? Applying bugfixes is optional. It should move to security-fix mode soon.
History
Date User Action Args
2022-04-11 14:57:38 admin set github: 60801
2015-09-21 15:03:28 zach.ware link issue25202 superseder
2012-12-20 19:11:20 benjamin.peterson set messages: +
2012-12-20 19:10:31 serhiy.storchaka set messages: +
2012-12-20 18:56:07 python-dev set messages: +
2012-12-20 18:52:06 serhiy.storchaka set messages: +
2012-12-20 18:26:55 serhiy.storchaka set messages: +
2012-12-20 18:24:22 python-dev set messages: +
2012-12-20 18:23:57 benjamin.peterson set messages: +
2012-12-20 18🔞53 serhiy.storchaka set files: + bufferedio_finally_close-2.7_2.patchmessages: +
2012-12-20 18:14:47 serhiy.storchaka set files: + bufferedio_finally_close-3.2_2.patchmessages: +
2012-12-20 17:55:28 python-dev set status: open -> closednosy: + python-devmessages: + resolution: fixedstage: patch review -> resolved
2012-12-20 17:37:53 serhiy.storchaka set files: + bufferedio_finally_close_2.patchmessages: +
2012-12-20 16:15:30 serhiy.storchaka set messages: +
2012-12-20 14:13:06 christian.heimes set nosy: + christian.heimes
2012-12-16 13:41:20 pitrou set nosy: + hynek
2012-12-15 18:01:09 asvetlov set nosy: + asvetlov
2012-12-05 19:22:41 jcea set nosy: + jcea
2012-12-03 12:28:51 serhiy.storchaka set files: + bufferedio_finally_close.patch
2012-12-03 12:27:52 serhiy.storchaka set files: - bufferedio_finally_close.patch
2012-12-03 12:26:00 serhiy.storchaka set files: + bufferedio_finally_close.patchnosy: + pitrou, benjamin.peterson, stutzbachmessages: + keywords: + patchstage: patch review
2012-12-03 09:24:50 serhiy.storchaka set messages: + components: + Extension Modules
2012-12-03 09:22:21 serhiy.storchaka set nosy: + serhiy.storchakamessages: + versions: + Python 2.7, Python 3.3, Python 3.4
2012-12-03 07:54:13 Arfrever set nosy: + Arfrever
2012-12-02 23:31:08 udoprog set messages: +
2012-12-02 23:27:35 udoprog set title: close not being called with context manager on IOError when device is full. -> file descriptor not being closed with context manager on IOError when device is full
2012-12-02 23:24:53 udoprog create