[Python-Dev] httplib and bad response chunking (original) (raw)

John J Lee jjl at pobox.com
Wed Aug 9 22:40:25 CEST 2006

On Sun, 30 Jul 2006, Greg Ward wrote: [...]

Did you look at the crude attempt at testing for this bug that I hacked into testhttplib.py? I posted it to bug #1486335 here:

http://sourceforge.net/tracker/download.php?groupid=5470&atid=105470&fileid=186245&aid=1486335 The idea is simple: put various chunked responses into strings and then feed those strings to HTTPConnection. The trouble is that StringIO does not behave the same as a real socket: where HTTPResponse fails one way reading from a real socket (eg. infinite loop), it fails differently (or not at all) reading from a StringIO. Makes testing with the FakeSocket class in testhttplib.py problematic.

There are always ways around unit testing problems, but...

Maybe the right way to test httplib is to spawn a server process (thread?) to listen on some random port, feed various HTTP responses at HTTPConnection/HTTPResponse, and see what happens. I'm not sure how to do that portably, though. Well, I'll see if I can whip up a Unix-y solution and see if anyone knows how to make it portable.

I think adding this kind of functional test is extremely valuable, given that we don't seem to have any for httplib at present (Lib/test/test_httplib.py does not send any network packets). Maybe you could add a basic smoke test that does a simple successful GET, while you're going about adding your chunked encoding regression test(s)?

Oh, wait: there is a functional test that uses the network, but it's at the end of httplib.py rather than being part of the test suite, and it follows the "Guru Watches Output" pattern :-)

I tried to add a test for urllib2 recently that relied on a python.org URL being set up, but the python.org guys are pretty overworked already and haven't had time to enable that yet -- so I think that simply from that point of view your run-your-own-server approach is better. Why does it need to be unix-y, though? We have SimpleHTTPServer. We're not trying to test the OS, so I don't see a problem with using loopback and a single process -- and that would keep test run times down. Um, except that MS OSes seem a bit odd re the loopback interface -- ISTR that, at least back with NT4, you just didn't get a loopback i/face unless you had an ethernet driver installed (back then, I was on dialup).

More unit tests would also be valuable, of course.

John

More information about the Python-Dev mailing list