[Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of d (original) (raw)

Gordon McMillan gmcm@hypernet.com
Fri, 11 Aug 2000 17:34:54 -0400

[Guido]

OK. So send() can do a partial write, but only on a stream connection. And most standard library code doesn't check for that condition, nor does (probably) much other code that used the standard library as an example. Worse, it seems that on some platforms send() never does a partial write (I couldn't reproduce it on Red Hat 6.1 Linux), so even stress testing may not reveal the lurking problem.

I'm quite sure you can force it with a non-blocking socket (on RH 5.2 64K blocks did it - but that's across a 10baseT ethernet connection).

Possible solutions:

1. Do nothing. Pro: least work. Con: subtle bugs remain.

Yes, but they're application-level bugs, even if they're in the std lib. They're not socket-support level bugs.

2. Fix all code that's broken in the standard library, and try to encourage others to fix their code. Book authors need to be encouraged to add a warning. Pro: most thorough. Con: hard to fix every occurrence, especially in 3rd party code.

As far as I can tell, Linux and Windows can't fail with the std lib code (it's all blocking sockets). Sam says BSDI could fail, and as I recall HPUX could too.

3. Fix the socket module to raise an exception when less than the number of bytes sent occurs. Pro: subtle bug exposed when it happens. Con: breaks code that did the right thing!

4. Fix the socket module to loop back on a partial send to send the remaining bytes. Pro: no more short writes. Con: what if the first few send() calls succeed and then an error is returned? Note: code that checks for partial writes will be redundant!

If you can exempt non-blocking sockets, either 3 or 4 (preferably 4) is OK. But if you can't exempt non-blocking sockets, I'll scream bloody murder. It would mean you could not write high performance socket code in Python (which it currently is very good for). For one thing, you'd break Medusa.

I'm personally in favor of (4), despite the problem with errors after the first call.

The sockets HOWTO already documents the problem. Too bad I didn't write it before that std lib code got written .

I still prefer leaving it alone and telling people to use makefile if they can't deal with it. I'll vote +0 on 4 if and only if it exempts nonblocking sockets.