[Python-Dev] microthreading vs. async io (original) (raw)

Adam Olsen rhamph at gmail.com
Mon Feb 26 15:02:44 CET 2007

On 2/25/07, Armin Rigo <arigo at tunes.org> wrote:

Hi Adam,

On Thu, Feb 15, 2007 at 06:17:03AM -0700, Adam Olsen wrote: > > E.g. have a wait(events = [], timeout = -1) method would be sufficient > > for most cases, where an event would specify > > I agree with everything except this. A simple function call would > have O(n) cost, thus being unacceptable for servers with many open > connections. Instead you need it to maintain a set of events and let > you add or remove from that set as needed. I just realized that this is not really true in the present context. If the goal is to support programs that "look like" they are multi-threaded, i.e. don't use callbacks, as I think is Joachim's goal, then most of the time the wait() function would be only called with a single event, rarely two or three, never more. Indeed, in this model a large server is implemented with many microthreads: at least one per client. Each of them blocks in a separate call to wait(). In each such call, only the events revelant to that client are mentioned. In other words, the cost is O(n), but n is typically 1 or 2. It is not the total number of events that the whole application is currently waiting on. Indeed, the scheduler code doing the real OS call (e.g. to select()) can collect the events in internal dictionaries, or in Poll objects, or whatever, and update these dictionaries or Poll objects with the 1 or 2 new events that a call to wait() introduces. In this respect, the act of calling wait() already means "add these events to the set of all events that need waiting for", without the need for a separate API for doing that.

That would depend on whether Joachim's wait() refers to the individual tasks' calls or the scheduler's call. I assumed it referred to the scheduler. In the basic form it would literally be select.select(), which has O(n) cost and often fairly large n.

-- Adam Olsen, aka Rhamphoryncus

More information about the Python-Dev mailing list