[Python-3000] Draft PEP for New IO system (original) (raw)

Daniel Stutzbach daniel.stutzbach at gmail.com
Sun Mar 4 22:34:44 CET 2007

On 3/1/07, Adam Olsen <rhamph at gmail.com> wrote:

Why do non-blocking operations need to use the same methods when they're clearly not the same semantics? Although long, .nonblockflush() would be explicit and allow .flush() to still block.

.nonblockflush() would be fine with me, but I don't think .flush() should block on a non-blocking object. To accomplish that, it would either have to be smart enough to switch the object into blocking mode, or internally use select().

How about .flush() write as much as it can, and throw an exception if not all of the bytes can be written to the device? (again, this would only come up when a user has set the underlying file descriptor to non-blocking mode)

I'm especially wary of infinite buffers. They allow a malicious peer to consume all your memory, DoSing the process or even the whole box if Linux's OOM killer doesn't kick in fast enough.

For a write-buffer, you start eating up memory only if an application is buffer-ignorant and tries to dump a massive amount of data to the socket all at once. A naive HTTP server implementation might do this by calling something like s.write(open(filename)). This isn't a DoS by a peer though, it's a local implementation problem.

For a read-buffer, you start eating up all of memory only if you call .read() with no arguments and the peer dumps a few gig on you. If you call read(n) to get as much data as you need, then the buffer class will only grab reasonably sized chunks from the network. Network applications don't normally call .read() without arguments, since they need to communicate both ways. If the object is an ordinary file, then DoS isn't so much of an issue and reading the whole files seems very reasonable.

I suppose for an Text object wrapped around a socket, .readline() could be dangerous if a malicious peer sends a few gig all on one line. That's a problem for the encoding layer to sort out, not the buffering layer though.

-- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC

More information about the Python-3000 mailing list