[Python-Dev] PEP 3156 - Asynchronous IO Support Rebooted (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Fri Dec 21 20:44:11 CET 2012

Hello,

To get the current event loop, use geteventloop(). This returns an instance of the EventLoop class defined below or an equivalent object. It is possible that geteventloop() returns a different object depending on the current thread, or depending on some other notion of context.

To set the current event loop, use seteventloop(eventloop), where eventloop is an instance of the EventLoop class or equivalent. This uses the same notion of context as geteventloop().

So can we instantiate an EventLoop directly and then call set_event_loop() with it? Or is the use case different?

- createtransport(protocolfactory, host, port, **kwargs). Creates a transport and a protocol and ties them together. Returns a Future whose result on success is a (transport, protocol) pair. Note that when the Future completes, the protocol's connectionmade() method has not yet been called; that will happen when the connection handshake is complete. When it is impossible to connect to the given host and port, the Future will raise an exception instead.

Optional keyword arguments: - family, type, proto, flags: Address familty, socket type, protcol, and miscellaneous flags to be passed through to getaddrinfo(). These all default to 0 except type which defaults to socket.SOCKSTREAM. - ssl: Pass True to create an SSL transport (by default a plain TCP is created). Or pass an ssl.SSLContext object to override the default SSL context object to be used. TBD: Should this be called createconnection()?

Either create_connection() or create_client(). create_transport() is wrong, since server transports wouldn't use that function.

I would favour create_client() if this function is also meant to support UDP (I know you haven't thought about UDP yet, but it is an important and common use case).

I have another question about that API: if I want to cancel the connection attempt after a given delay, how do I do that? If I call cancel() on the future, does it cancel the connect() call?

As for SSL, there are security issues with having a "default SSL context" (notably, any decent client use of SSL must check the server certificate against an appropriate set of CAs). It's much better to force users to pass a context explicitly. Choosing default settings should only be for higher-level APIs like urllib.request.

(btw, don't you mean that family defaults to AF_INET?)

If executor is None, a default ThreadPoolExecutor with 5 threads is used

Is it because Twisted's thread pool has minThreads=5? :)

The transport is free to buffer the bytes, but it must eventually cause the bytes to be transferred to the entity at the other end, and it must maintain stream behavior. That is, t.write(b'abc'); t.write(b'def') is equivalent to t.write(b'abcdef')

I think this is a bad idea. The kernel's network stack should do the buffering (and choose appropriate algorithms for that), not the user-level framework. The transport should write the bytes as soon as the fd is ready for writing, and it should write the same chunks as given by the user, not a concatenation of them.

Besides, it would be better if transports weren't automatically streaming transports. There are connected datagram protocols, such as named pipes under Windows (multiprocessing already uses non-blocking Windows named pipes).

Proposal: let the transport call protocol.pause() and protocol.resume() if they exist; if they don't exist, the protocol doesn't support flow control.

+1. The Protocol base class can provide default no-op implementations.

TBD: Discuss whether user code needs to do anything to make sure that protocol and transport aren't garbage-collected prematurely.

The transport should be tied to the event loop as long as the connection holds, and the protocol will hold to the transport.

TBD: Need an interface to wait for the first of a collection of Futures.

Have you looked at Twisted's DeferredList? http://twistedmatrix.com/documents/12.1.0/api/twisted.internet.defer.DeferredList.html

I think par() could take a keyword-only argument to specify you want the callback to be triggered on the first result (and perhaps being able to choose between "the first success result" and "the first success or error result").

A trick used by Richard Oudkerk in the tulip project's proactor branch makes calls like recv() either return a regular result or raise a Future. The caller (likely a transport) must then write code like this:

Isn't it a case of premature optimization? If we want to keep this, there should be a nicer API, perhaps like Twisted's maybeDeferred: http://twistedmatrix.com/documents/current/api/twisted.internet.defer.html#maybeDeferred

We might also introduce explicit locks (though these will be a bit of a pain to use, as we can't use the with lock: block syntax).

I don't understand why you couldn't use "with lock" in a coroutine. Am I misunderstanding something?

Is it reasonable to map write(), writelines(), datareceived() to single datagrams?

Well, at least that's how Twisted does it (not sure about writelines()).

Regards

Antoine.

More information about the Python-Dev mailing list