[Python-Dev] Fw: SocketServer and makefile() [from comp.lang.python] (original) (raw)

Guido van Rossum guido@beopen.com
Thu, 31 Aug 2000 15:48:16 -0500

I ran into this same problem on the client side.

The server does a makefile() so that it can do readline() to fetch the HTTP request line and then the MIME headers. The problem is that if you do something like: f = sock.makefile() line = f.readline() data = sock.recv(1000) You're screwed if you have buffering enabled. "f" will read in a bunch of data -- past the end of the line. That data now sits inside f's buffer and is not available to the sock.recv() call. If you forget about sock and just stick to f, then you'd be okay. But SocketServer and/or BaseHTTPServer doesn't -- it uses both objects to do the reading. Solution? Don't use rfile for reading, but go for the socket itself. Or revamp the two classes to forget about the socket once the files (wfile and rfile) are created. The latter might not be possible, tho.

I was about to say that you have it backwards, and that you should only use rfile & wfile, when I realized that CGIHTTPServer.py needs this! The subprocess needs to be able to read the rest of the socket, for POST requests. So you're right.

Solution? The buffer size should be an instance or class variable. Then SocketServer can set it to buffered by default, and CGIHTTPServer can set it to unbuffered.

Dunno why the unbuffered reading would be slow. I'd think it would still read large chunks at a time when you request it.

System call overhead? I had the same complaint about Windows, where apparently winsock makes you pay more of a performance penalty than Unix does in the same case.

--Guido van Rossum (home page: http://www.pythonlabs.com/~guido/)