| msg207368 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2014-01-05 10:22 |
| On old FreeBSD platforms (FreeBSD 6), concurrent.futures cannot be used and fail with an ImportError because of a limit on POSIX semaphores: see issue #10798. (See also maybe issue #10348 "multiprocessing: use SysV semaphores on FreeBSD" closed as WONTFIX.) For asyncio, the test is currently skipped on this platform: see issue #19635. (See also maybe issue #19295 "Make asyncio work without threads" closed as WONTFIX.) I propose to implement a simple fallback using an synchronous executor (block until we get the result) which would be used by getaddrinfo() and getnameinfo() methods of EventLoop. This issue is also motivated by my Trollius project, a backport of asyncio for Python 2.7, where concurrent.futures is not available. An optional backport, the "futures" project, can be used: https://pypi.python.org/pypi/futures Attached patch is a work-in-progress, it removes the dependency to concurrent.futures but doesn't implement the synchronous executor yet. It's the changes I wrote in Trollius 0.1. https://pypi.python.org/pypi/trollius |
|
|
| msg207377 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2014-01-05 14:43 |
| That doesn't sound like a very good idea, only to support FreeBSD 6. Synchronous execution blocks the event loop waiting for I/O, which basically destroys the point of having an event loop. Let users of FreeBSD 6 upgrade their systems instead, IMHO. |
|
|
| msg207388 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2014-01-05 18:11 |
| 2014/1/5 Antoine Pitrou <report@bugs.python.org>: > That doesn't sound like a very good idea, only to support FreeBSD 6. Well, if the patch is rejected here, I will propose it in Tulip, or I will keep it only in Trollius (the port on Python 2.7). The idea of this issue was discussed on the python-tulip group and approved by Guido van Rossum: https://groups.google.com/forum/#!msg/python-tulip/1CbjmZTXINM/pR-_CVga7bUJ > Synchronous execution blocks the event loop waiting for I/O, which basically destroys the point of having an event loop. asyncio only uses the executor to resolve host names. Resolution is usually fast, and you may only resolve a name once at startup. But I agree that it's slower than supporting asynchronous resolution. I hope that asyncio will get its down asynchronous DNS resolver! There are existing projects: https://pyuv.readthedocs.org/en/release-0.6.1/dns.html (Asynchronous DNS resolver using c-ares for pyuv) http://code.google.com/p/asyncdns/ https://code.google.com/p/adns-python/ (Python interface to GNU adns asynchronous resolver library) etc. > Let users of FreeBSD 6 upgrade their systems instead, IMHO. Well, sometimes you don't have the choice of your OS :-) |
|
|
| msg207392 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2014-01-05 19:06 |
| Here is an implementation of a synchrounous executor: https://bitbucket.org/haypo/trollius/commits/1bc2c23854e6717007a5a6df42470afa49c79b99 The changes are larger than what I expected. On Python 3.4, if concurrent.futures is only missing on FreeBSD 6, I agree that it's ok to ask users to upgrade their OS. @Guido: Are you interested by this change, since Tulip requires at least Python 3.3 which has also concurrent.futures? The fallback is probably only needed by the Trollius project in practice. |
|
|
| msg207394 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2014-01-05 19:16 |
| > Here is an implementation of a synchrounous executor: > https://bitbucket.org/haypo/trollius/commits/1bc2c23854e6717007a5a6df42470afa49c79b99 Where is executor.py? |
|
|
| msg207395 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2014-01-05 19:23 |
| > Where is executor.py? Hum, maybe I forgot to push it :-) https://bitbucket.org/haypo/trollius/src/tip/asyncio/executor.py?at=trollius Code of the synchronous executor: https://bitbucket.org/haypo/trollius/src/8f28756d63c2c9f09c3a7c1df420796eca9ff08b/asyncio/executor.py?at=trollius#cl-60 I only defined methods used by asyncio. I didn't try to implement the full API (ex: it's not possible to cancel a Future). |
|
|
| msg207396 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2014-01-05 19:27 |
| > https://bitbucket.org/haypo/trollius/src/8f28756d63c2c9f09c3a7c1df420796eca9ff08b/asyncio/executor.py?at=trollius#cl-60 Oh, there was a forgotten "raise" instruction. I added it to test the exception case, but it looks like this case is not tested by asyncio test suite. (I just removed the "raise" instruction.) |
|
|
| msg207685 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2014-01-08 14:18 |
| The synchronous executor has been implemented in Trollius 0.1.1. I now agree with Antoine Pitrou that it's overkill to implement in CPython since only one old platform don't support concurrent.futures. On FreeBSD 6: increase the limit of POSIX semaphore, or upgrade to a more recent FreeBSD version, or use Trollius. |
|
|