[Python-Dev] threadsafe patch for asynchat (original) (raw)

Tim Peters tim.peters at gmail.com
Wed Feb 8 06:15:41 CET 2006

[Josiah Carlson]

... Back to the topic that Guido was really complaining about: Zope + asyncore. I don't doubt that getting Zope to play nicely with asyncore was difficult,

It's more that mixing asyncore with threads is a bloody nightmare, and ZEO and Zope both do that. Zope (but not ZEO) goes on to mix threads with asynchat too. In addition, ZEO makes life much harder than should be necessary by running in two different modes and auto-switching between them, depending on whether "the app" is or is not running an asyncore mainloop itself. In order to detect when "the app" fires up an asyncore mainloop, ZEO monkey-patches asyncore's loop() function and physically replaces it with its own loop() function. It goes downhill from there.

Guido's memories are partly out of date now: ZEO used to replace a lot more of asyncore than it does now, because of bugs in the asyncore distributed with older Python versions. The needs for that went away little by little over the years, but the code in ZEO stuck around much longer. ZEO's ThreadedAsync/LoopCallback.py is much smaller now (ZODB 3.6) than Guido remembers.

For a brief while, I even ripped out ZEO's monkey-patching of Python's asyncore loop(), but it turned out that newer code in Zope3 (but not Zope2) relied on, in turn, poking values into ZEO's module globals to cause ZEO's loop() replacement to shut down (that's the kind of "expedient" joy you get when mixing asyncore with threads).

Every piece of it remains "underdocumented" and, IMO, highly obscure.

but it begs the questions: what would have been done if asyncore didn't exist,

Who knows? What would python-dev be like if you didn't exist :-)?

and why wasn't that done instead of trying to play nicely with asyncore?

Bugs and "missing features" in asyncore. For ZEO's purposes, if I had designed it, I expect it would have used threads (without asyncore). However, bits of code still sitting around suggest that it was at least the intent at one time that ZEO be able to run without threads at all. That's certainly not possible now.

If you look at asyncore's revision history, you'll note that Jeremy and Guido made many changes when they worked at Zope Corp. Those largely reflect the history of moving ZEO's asyncore monkey-patches into the Python core.

BTW, if you don't use ZEO, I believe it's possible to run Zope3 without asyncore (you can use Twisted in Zope3 instead).

More information about the Python-Dev mailing list