Issue 23752: Cleanup io.FileIO - Python tracker (original) (raw)

Created on 2015-03-23 16:51 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
fileio_drop_dup_fstat.patch vstinner,2015-03-23 20:46 review
Messages (10)
msg239045 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-23 16:51
While reviewing a Python implementation of io.FileIO (_pyio.FileIO) in the issue #21859, many issues were found in the C implementation. I open an issue to fix them. We may fix them before or after the Python implementation is merged.
msg239063 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-23 20:46
fileio_drop_dup_fstat.patch: Remove the duplicate call to fstat() in fileio.c. Tests pass, invalid FD are still detected since there is a second call to fstat() which also raises OSError(EBADF) if fstat(fd) fails.
msg239066 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-23 20:54
The previous change related to fstat() is the changeset 3b5279b5bfd1 from the issue #21679. The changeset introduces the private _blksize attribute. The strange thing is that the changelog is: "Issue #21679: Prevent extraneous fstat() calls during open(). Patch by Bohuslav Kabrda." fstat() was called twice and it is still called twice. Maybe I missed something.
msg239070 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-23 21:18
See also issue #21861: "io class name are hardcoded in reprs". In my review of _pyio.FileIO, I asked if it would be possible to use __qualname__ in __getstate__().
msg239548 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-30 01:22
New changeset bc2a22eaa0af by Victor Stinner in branch 'default': Issue #23752: When built from an existing file descriptor, io.FileIO() now only https://hg.python.org/cpython/rev/bc2a22eaa0af
msg239577 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-30 08:19
Hum, _Py_fstat() is not used correctly in _io.FilIO: it uses errno instead of GetLastError() to raise the OSError. It's too hard to use the new _Py_fstat() function directly, I modified it to raise directly the exception. So the caller doesn't have to use #ifdef MS_WINDOWS to raise the exception correctly. I added _Py_fstat_noraise() when it's not possible to use an exception. http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/5978/steps/test/logs/stdio ====================================================================== FAIL: test_fdopen (test.test_os.TestInvalidFD) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_os.py", line 1418, in check f(support.make_bad_fd(), *args) File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\os.py", line 1028, in fdopen return io.open(fd, *args, **kwargs) OSError: [Errno 0] Error During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_os.py", line 1411, in helper self.check(getattr(os, f)) File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_os.py", line 1420, in check self.assertEqual(e.errno, errno.EBADF) AssertionError: 0 != 9
msg239578 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-30 08:20
New changeset 128f92ae8bae by Victor Stinner in branch 'default': Issue #23752: _Py_fstat() is now responsible to raise the Python exception https://hg.python.org/cpython/rev/128f92ae8bae
msg239588 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-30 09:41
The changeset 128f92ae8bae also changed _Py_fstat() to release the GIL when calling fstat(), I forgot to mention it in the changeset. test_socket now hangs on AMD64 Snow Leop 3.x, I don't know yet if it's related: http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.x/builds/2870/steps/test/logs/stdio [393/393] test_socket Timeout (1:00:00)! Thread 0x0000000104fef000 (most recent call first): File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/socket.py", line 288 in _sendfile_use_sendfile File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_socket.py", line 5131 in _testOffset File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_socket.py", line 278 in clientRun Thread 0x00007fff71296cc0 (most recent call first): File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/threading.py", line 293 in wait File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/threading.py", line 556 in wait File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_socket.py", line 262 in _tearDown File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/case.py", line 580 in run File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/case.py", line 625 in __call__ File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 122 in run File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 84 in __call__ File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 122 in run File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 84 in __call__ File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/runner.py", line 176 in run File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/support/__init__.py", line 1773 in _run_suite File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/support/__init__.py", line 1807 in run_unittest File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_socket.py", line 5340 in test_main File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/regrtest.py", line 1280 in runtest_inner
msg239589 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-30 09:44
Oh, there was another hang before, so it's probably not related: http://buildbot.python.org/all/builders/AMD64%20Snow%20Leop%203.x/builds/2852/steps/test/logs/stdio [393/393] test_ssl Timeout (1:00:00)! Thread 0x0000000102d81000 (most recent call first): File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_ssl.py", line 2832 in serve File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/threading.py", line 871 in run File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/threading.py", line 923 in _bootstrap_inner File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/threading.py", line 891 in _bootstrap Thread 0x00007fff71296cc0 (most recent call first): File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/ssl.py", line 628 in do_handshake File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/ssl.py", line 983 in do_handshake File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/ssl.py", line 747 in __init__ File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/ssl.py", line 1064 in wrap_socket File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/case.py", line 162 in handle File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/case.py", line 1238 in assertRaisesRegex File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/test_ssl.py", line 2851 in test_handshake_timeout File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/case.py", line 577 in run File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/case.py", line 625 in __call__ File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 122 in run File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 84 in __call__ File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 122 in run File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/suite.py", line 84 in __call__ File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/unittest/runner.py", line 176 in run File "/Users/buildbot/buildarea/3.x.murray-snowleopard/build/Lib/test/support/__init__.py", line 1773 in _run_suite
msg240193 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-04-06 23:38
Buildbot issues were unrelated and have been fixed. I close the issue.
History
Date User Action Args
2022-04-11 14:58:14 admin set github: 67940
2015-04-06 23:38:35 vstinner set status: open -> closedresolution: fixedmessages: +
2015-03-30 09:44:58 vstinner set messages: +
2015-03-30 09:41:18 vstinner set messages: +
2015-03-30 08:20:08 python-dev set messages: +
2015-03-30 08:19:58 vstinner set status: closed -> opennosy: + holdenwebmessages: + resolution: fixed -> (no value)
2015-03-30 01:23:26 vstinner set status: open -> closedresolution: fixed
2015-03-30 01:22:53 python-dev set nosy: + python-devmessages: +
2015-03-23 22:57:03 piotr.dobrogost set nosy: + piotr.dobrogost
2015-03-23 21🔞52 vstinner set messages: +
2015-03-23 20:54:21 vstinner set messages: +
2015-03-23 20:46:59 vstinner set files: + fileio_drop_dup_fstat.patchkeywords: + patchmessages: +
2015-03-23 16:51:05 vstinner create