| msg57864 - (view) |
Author: Guido van Rossum (gvanrossum) *  |
Date: 2007-11-27 01:23 |
| See e.g.: http://www.python.org/dev/buildbot/3.0/ppc%20Debian%20unstable%203.0/builds/303/step-test/0 Note how it fails the first time and passes on the re-run. I've seen this before (just not on any of my own systems). I've also seen it fail twice (on buildbot machines). |
|
|
| msg64566 - (view) |
Author: Ralf Schmitt (schmir) |
Date: 2008-03-26 21:25 |
| The current buildbot has errors similar to this one (I assume): Exception happened during processing of request from ('127.0.0.1', 53126) Traceback (most recent call last): File "/Users/ralf/trunk/Lib/SocketServer.py", line 281, in _handle_request_noblock self.process_request(request, client_address) File "/Users/ralf/trunk/Lib/SocketServer.py", line 307, in process_request self.finish_request(request, client_address) File "/Users/ralf/trunk/Lib/SocketServer.py", line 320, in finish_request self.RequestHandlerClass(request, client_address, self) File "/Users/ralf/trunk/Lib/SocketServer.py", line 615, in __init__ self.handle() File "/Users/ralf/trunk/Lib/BaseHTTPServer.py", line 318, in handle self.handle_one_request() File "/Users/ralf/trunk/Lib/BaseHTTPServer.py", line 301, in handle_one_request self.raw_requestline = self.rfile.readline() File "/Users/ralf/trunk/Lib/socket.py", line 369, in readline data = self._sock.recv(self._rbufsize) error: [Errno 35] Resource temporarily unavailable ---------------------------------------- The problem is that the test calls serv.socket.settimeout(3) in the http_server function. This implicitly sets the server socket to nonblocking state. The accept call then returns a socket object, which - is blocking on OS X 10.4 ppc - is nonblocking on linux I can easily reproduce that on my mac mini g4 with python 2.6. |
|
|
| msg64567 - (view) |
Author: Ralf Schmitt (schmir) |
Date: 2008-03-26 22:01 |
| I just double checked with the following program: #! /usr/bin/env python import os import fcntl import socket def isnonblocking(fd): return bool(fcntl.fcntl(fd, fcntl.F_GETFL, 0) & os.O_NONBLOCK) def main(): s=socket.socket() s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) s.bind(("0.0.0.0", 8000)) s.listen(5) s.settimeout(30) print "server isnonblocking:", isnonblocking(s.fileno()) client, addr = s.accept() print "client isnonblocking:", isnonblocking(client.fileno()) if __name__=="__main__": main() on my g4 mac it prints: ~/ python serv.py ralf@mini ok server isnonblocking: True client isnonblocking: True on linux: ~/ python mini/serv.py ralf@rat64 ok server isnonblocking: True client isnonblocking: False |
|
|
| msg64568 - (view) |
Author: Ralf Schmitt (schmir) |
Date: 2008-03-26 22:08 |
| now that I see that the buildbot was running on ppc *Debian* I'm not quite sure if we're talking about the same issue. |
|
|
| msg64569 - (view) |
Author: Alan McIntyre (alanmcintyre) * |
Date: 2008-03-26 22:10 |
| It's my fault the xmlrpc tests try to use non-blocking sockets. That got added because sometimes failing tests would just sit there with the server blocking until the entire test process got killed for running too long. There are some tests that are skipped in test_xmlrpc because of (apparent) Windows socket quirks; should they also be skipped for OS X PPC, or should the flaky tests just be scrapped? When I was last working on this I couldn't come up with a better way to run these tests, so unless somebody can suggest a new approach I'm just left with recommending the skip & scrap options as the only way to stop the flakiness. |
|
|
| msg64570 - (view) |
Author: Ralf Schmitt (schmir) |
Date: 2008-03-26 22:18 |
| No, please do not disable them. I'm not quite sure what to do, but apparently these sockets returned from accept should be turned into blocking sockets. I just do not know, where this should happen. I think that this could even be done inside the accept call (which then might break some code). At least it should be done in the BaseHTTPServer code, which apparently cannot handle nonblocking sockets. |
|
|
| msg64571 - (view) |
Author: Ralf Schmitt (schmir) |
Date: 2008-03-26 22:54 |
| With the following diff, test_xmlrpc.py passes without problems. Like I said, someone else should decide where to turn on blocking mode. I now think that it should be in the socket.accept (i.e. in the C code) at least for unix platforms. Those who really want a nonblocking socket, will most probably call that setblocking(0) anyway (or their program is broken on linux, which returns blocking sockets by default). --- a/Lib/SocketServer.py Wed Mar 26 22:41:36 2008 +0100 +++ b/Lib/SocketServer.py Wed Mar 26 23:48:13 2008 +0100 @@ -441,8 +441,10 @@ May be overridden. """ - return self.socket.accept() - + r= self.socket.accept() + r[0].setblocking(1) + return r + def close_request(self, request): """Called to clean up an individual request.""" request.close() |
|
|
| msg64616 - (view) |
Author: Neal Norwitz (nnorwitz) * |
Date: 2008-03-28 06:39 |
| Ugh. The manpage for accept on Ubuntu 6.10 says: """ On Linux, the new socket returned by accept() does not inherit file status flags such as O_NONBLOCK and O_ASYNC from the listening socket. This behaviour differs from the canonical BSD sockets implementation. Portable programs should not rely on inheritance or non-inheritance of file status flags and always explicitly set all required flags on the socket returned from accept(). """ http://msdn2.microsoft.com/en-us/library/aa450277.aspx says that Windows (CE, but I assume all variants) are like BSD in that they inherit attributes. """The newly created socket is the socket that will handle the actual connection and has the same properties as socket s, including the asynchronous events registered with the WSAEventSelect function.""" I assume that means blocking behavior. I checked in r61993 which should fix the immediate problem with test_xmlrpc. I wonder if we should change socket to do the same thing for all platforms. |
|
|
| msg69720 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2008-07-15 21:26 |
| Does anybody still care about this? |
|
|
| msg69724 - (view) |
Author: Ralf Schmitt (schmir) |
Date: 2008-07-15 22:25 |
| I still think that blocking mode should be turned on in socket.accept. settimeout implicitly sets non-blocking mode (which might be a bit surprising). The sockets returned from accept() being in non-blocking mode is surprising (as this bug shows). |
|
|
| msg70750 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2008-08-05 16:09 |
| Without a patch this will not improve. |
|
|
| msg70759 - (view) |
Author: Ralf Schmitt (schmir) |
Date: 2008-08-05 19:34 |
| I would say without a decision on what to do, there will be no patch :) I can write a patch, which unconditionally turns on blocking mode for sockets returned from accept (on unix-platforms). Would that be fine? |
|
|
| msg114599 - (view) |
Author: Georg Brandl (georg.brandl) * |
Date: 2010-08-21 23:06 |
| I've not seen special failures of test_xmlrpc lately. |
|
|
| msg118490 - (view) |
Author: Ralf Schmitt (schmir) |
Date: 2010-10-13 00:09 |
| The tests now pass, but the underlying issue hasn't been fixed! |
|
|