| msg211095 - (view) |
Author: Florent Xicluna (flox) *  |
Date: 2014-02-12 15:13 |
| I had this sporadic traceback in a project: File "test.py", line 62, in result = do_lqs(client, str(dnvn)) File "test.py", line 25, in do_lqs qualif_service_id = client.create('ti.qualif.service', {}) File "/srv/openerp/.buildout/eggs/ERPpeek-1.4.5-py2.6.egg/erppeek.py", line 894, in wrapper return self.execute(obj, method, *params, **kwargs) File "/srv/openerp/.buildout/eggs/ERPpeek-1.4.5-py2.6.egg/erppeek.py", line 636, in execute res = self._execute(obj, method, *params) File "/srv/openerp/.buildout/eggs/ERPpeek-1.4.5-py2.6.egg/erppeek.py", line 361, in wrapper = lambda s, *args: s._dispatch(name, args) File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request verbose=self.__verbose File "/usr/lib/python2.6/xmlrpclib.py", line 1235, in request self.send_content(h, request_body) File "/usr/lib/python2.6/xmlrpclib.py", line 1349, in send_content connection.endheaders() File "/usr/lib/python2.6/httplib.py", line 908, in endheaders self._send_output() File "/usr/lib/python2.6/httplib.py", line 780, in _send_output self.send(msg) File "/usr/lib/python2.6/httplib.py", line 739, in send self.connect() File "/usr/lib/python2.6/httplib.py", line 1112, in connect sock = socket.create_connection((self.host, self.port), self.timeout) File "/usr/lib/python2.6/socket.py", line 561, in create_connection raise error, msg socket.error: [Errno 4] Interrupted system call It seems that the EINTR should be caught by the standard library in all cases: http://bugs.python.org/issue1628205 But it's not the case for the "socket.create_connection" method (verified in 3.3 and 2.7 source code). |
|
|
| msg211128 - (view) |
Author: Charles-François Natali (neologix) * |
Date: 2014-02-13 05:13 |
| > It seems that the EINTR should be caught by the standard library in all cases: > http://bugs.python.org/issue1628205 Yes, it should. > But it's not the case for the "socket.create_connection" method (verified in 3.3 and 2.7 source code). It's not the case for *almost all* syscalls. See http://bugs.python.org/issue18885 |
|
|
| msg218831 - (view) |
Author: (tholzer) |
Date: 2014-05-20 03:12 |
| We encountered the same problem, this is in the context of using PyQt (specifically QProcess) or twisted. They both rely on SIGCHLD for their notification framework. I've attached a httplib EINTR patch for 2.6.4 & 2.7.3. |
|
|
| msg218833 - (view) |
Author: (tholzer) |
Date: 2014-05-20 04:09 |
| Here is a reproducible test case: import threading import signal import os import httplib def killer(): while 1: os.kill(os.getpid(), signal.SIGINT) def go(): signal.signal(signal.SIGINT, lambda x,y: None) thread = threading.Thread(target=killer) thread.start() while 1: connection = httplib.HTTPConnection("localhost:80") connection.connect() connection.close() if __name__ == '__main__': go() Which gives: Traceback (most recent call last): File "./repro1.py", line 22, in go() File "./repro1.py", line 18, in go connection.connect() File ".../lib/python2.7/httplib.py", line 757, in connect self.timeout, self.source_address) File ".../lib/python2.7/socket.py", line 571, in create_connection raise err socket.error: |
|
|
| msg218836 - (view) |
Author: Charles-François Natali (neologix) * |
Date: 2014-05-20 05:41 |
| As said in a previous comment, we don't want to have EINTR handling code everywhere. The right way to do this is to handle it at the syscall level. |
|
|
| msg218841 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2014-05-20 11:39 |
| @tholzer, to clarify Charles-François's comment: EINTR should be handled in socket.connect() and socket.getaddrinfo() (the two syscalls called by create_connection(), AFAIR). |
|
|
| msg219121 - (view) |
Author: (tholzer) |
Date: 2014-05-26 02:58 |
| No problem, I've attached a patch for socket.py for Python 2.7.3. A few notes: getaddrinfo (and gethostbyname, etc.) are already immune to this bug, so I've just fixed the connect() call. The socket does need to be closed after EINTR, otherwise a EINPROGRESS might get returned on subsequent connect() calls. |
|
|
| msg219122 - (view) |
Author: (tholzer) |
Date: 2014-05-26 03:42 |
| I've also attached a potential patch for the C module Modules/socketmodule.c inside internal_connect(). A few notes: This seems to work both without time-out and with time-out sockets (non-blocking). One concern would be a signal storm prolonging the operation beyond the time-out. Should we keep track of the actual time taken in this loop and check it against the 'timeout' parameter ? Also, I don't think we can call PyErr_CheckSignals() in this context. Does this need to happen at all ? |
|
|
| msg219337 - (view) |
Author: Syou Ei (meishao) * |
Date: 2014-05-29 06:02 |
| http://bugs.python.org/issue21602 The smtplib.py also has the same problem. The EINTR cannot be handled properly. |
|
|
| msg219340 - (view) |
Author: Syou Ei (meishao) * |
Date: 2014-05-29 08:30 |
| @neologix, May I attach the patch file of smtplib.py for review? |
|
|
| msg219343 - (view) |
Author: Florent Xicluna (flox) * |
Date: 2014-05-29 09:26 |
| @meishao Previous comments answer your question : http://bugs.python.org/issue20611#msg218836 http://bugs.python.org/issue20611#msg218841 |
|
|
| msg219344 - (view) |
Author: Syou Ei (meishao) * |
Date: 2014-05-29 10:18 |
| @flox Thank you for your comment. So we just only modify the socket.py to handle the system level call, is it right? Please let me attach the patch file of socket.py for 2.7.2. |
|
|
| msg219371 - (view) |
Author: (tholzer) |
Date: 2014-05-29 22:16 |
| Oops, I missed a break statement at the end of socket_2.7.3_eintr_patch.py. I've fixed this now in the attached patch. @meishao Could you please also update your socket_2_7_2_patch.py and add the missing break statement ? |
|
|
| msg219375 - (view) |
Author: Syou Ei (meishao) * |
Date: 2014-05-30 01:03 |
| @tholzer I've updated socket_2_7_2_patch.py and added the missing break statement. |
|
|
| msg219376 - (view) |
Author: (tholzer) |
Date: 2014-05-30 01:37 |
| And a test case for smtplib: import threading import signal import os import smtplib def go(): running = True pid = os.getpid() def killer(): while running: os.kill(pid, signal.SIGINT) signal.signal(signal.SIGINT, lambda x,y: None) thread = threading.Thread(target=killer) thread.start() while 1: try: smtplib.SMTP('localhost') except Exception, ex: running = False raise if __name__ == '__main__': go() Fails with: socket.error: [Errno 4] Interrupted system call |
|
|
| msg219668 - (view) |
Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2014-06-03 07:44 |
| Something like the patch i'm attaching to socketmodule.c is what I would prefer. I haven't looked at or tried tests for it yet. |
|
|
| msg223890 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2014-07-24 20:30 |
| The issue is just an example of the main issue #18885 which proposes to retry interrupted syscalls. I hesitate to mark it as duplicate, but it contains an interesting patch. |
|
|
| msg235544 - (view) |
Author: Martin Panter (martin.panter) * |
Date: 2015-02-08 03:38 |
| See also PEP 475 and Issue 23285 for the general fix in Python 3 |
|
|
| msg240194 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2015-04-06 23:53 |
| This issue was fixed in Python 3.5 as part of the PEP 475: see issue #23618 which modified socket.socket.connect() to handle EINTR. It's not as simple as retrying connect() in a loop. You must respect the socket timeout (it's better to have a monotonic clock here, it's now always the case in Python 3.5). When connect() returns EINTR, the connection runs asynchronously, you have to call select() to wait until the connection completes or fails. Then you have to call getsockopt() to get the socket error code. In Python 3.5, socket.socket.connect() still raises InterruptedError if the socket is non-blocking: https://docs.python.org/dev/library/socket.html#socket.socket.connect -connect-eintr-gps01.diff calls again connect() if connect() failed with EINTR. According to the issue #23618, it might work on some platforms, but it's not portable. For Python 2.7 and 3.4, instead of fixing socket.socket.connect(), which requires complex code, we may only workaround the issue in create_connection(). If connect() raises OSError(EINTR), drop the socket and retry with a fresh connection in a loop until the connection completes or raises a different exception. (And do that for each address.) |
|
|
| msg240232 - (view) |
Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2015-04-07 22:13 |
| i'm moving this to the more recent issue as i like the patch in that one better. |
|
|