| msg38987 - (view) |
Author: James Rucker (jamesrucker) |
Date: 2002-02-13 23:48 |
| xmlrpclib.Transport.parse_response() (called from xmlrpclib.Transport.request()) is exhibiting poor performance - approx. 10x slower than expected. I investigated based on using a simple app that sent a msg to a server, where all the server did was return the message back to the caller. From profiling, it became clear that the return trip was taken 10x the time consumed by the client->server trip, and that the time was spent getting things across the wire. parse_response() reads from a file object created via socket.makefile(), and as a result exhibits performance that is about an order of magnitude worse than what it would be if socket.recv() were used on the socket. The patch provided uses socket.recv() when possible, to improve performance. The patch provided is against revision 1.15. Its use provides performance for the return trip that is more or less equivalent to that of the forward trip. |
|
|
| msg38988 - (view) |
Author: Guido van Rossum (gvanrossum) *  |
Date: 2002-02-28 23:23 |
| Logged In: YES user_id=6380 Fredrik, does this look OK to you? |
|
|
| msg38989 - (view) |
Author: Fredrik Lundh (effbot) * |
Date: 2002-03-01 14:34 |
| Logged In: YES user_id=38376 looks fine to me. I'll merge it with SLAB changes, and will check it into the 2.3 codebase asap. (we probably should try to figure out why makefile causes a 10x slowdown too -- xmlrpclib isn't exactly the only client library reading from a buffered socket) |
|
|
| msg38990 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-03-01 16:14 |
| Logged In: YES user_id=6380 My guess makefile() isn't buffering properly. This has been a long-standing problem on Windows; I'm not sure if it's an issue on Unix. |
|
|
| msg38991 - (view) |
Author: Fredrik Lundh (effbot) * |
Date: 2002-03-17 13:30 |
| Logged In: YES user_id=38376 James, what platform(s) did you use? I'm not sure changing the parse_response() interface is a good idea, but if this is a Windows-only problem, there may be a slightly cleaner way to get the same end result. |
|
|
| msg38992 - (view) |
Author: James Rucker (jamesrucker) |
Date: 2002-03-17 16:13 |
| Logged In: YES user_id=351540 The problem was discovered under FreeBSD 4.4. |
|
|
| msg38993 - (view) |
Author: Fredrik Lundh (effbot) * |
Date: 2002-03-19 09:24 |
| Logged In: YES user_id=38376 What server did you use? In all my test setups, h._conn.sock is None at the time parse_response is called... |
|
|
| msg38994 - (view) |
Author: James Rucker (jamesrucker) |
Date: 2002-03-19 19:47 |
| Logged In: YES user_id=351540 HTTPConnection.getresponse() will close the socket and set self.sock to null after instantiating response_class (by default, this is HTTPResponse; note that HTTPResponse does a makefile() and stores the result in self.fp) iff the newly created response class instance's 'will_close' attribute is true. My server is setting the Keep-alive header with a value of 1 (it is based on xmlrpcserver.py), which causes will_close to evaluate to false. In your case, I'm presuming that will_close is being evaluated as false and thus the socket (accessed via h._conn.sock) has been set to . Note that when I removed the Keep-alive header, I witness the behaviour you're seeing. Thus, it seems that as it stands, the beneift of the change will only be realized if Keep-alive is set or HTTP/1.1 is used (and Keep-alive is either not specified or is set to non-zero). The following from httplib.py shows and explains how 'will_close' will be set (from httplib.py): conn = self.msg.getheader('connection') if conn: conn = conn.lower() # a "Connection: close" will always close the connection. if we # don't see that and this is not HTTP/1.1, then the connection will # close unless we see a Keep-Alive header. self.will_close = conn.find('close') != -1 or \ ( self.version != 11 and \ not self.msg.getheader('keep-alive') ) else: # for HTTP/1.1, the connection will always remain open # otherwise, it will remain open IFF we see a Keep-Alive header self.will_close = self.version != 11 and \ not self.msg.getheader('keep-alive') |
|
|
| msg38995 - (view) |
Author: Tim Peters (tim.peters) * |
Date: 2004-03-21 19:27 |
| Logged In: YES user_id=31435 It's been 2 years since this report was updated. Any chance of resolving it? If not, please close it. |
|
|
| msg38996 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2004-08-18 11:14 |
| Logged In: YES user_id=21627 This was integrated into xmlrpclib.py 1.19 long ago; closing it as accepted. |
|
|