| msg325350 - (view) |
Author: Nic Watson (jnwatson) |
Date: 2018-09-14 15:27 |
| Summary: essentially asyncio.add_signal_handler doesn't work when called off the main thread. One might consider this a documentation failure. (Note: there's also a bigger issue with cleanup, which I'll submit separately) Exception in thread Thread-1: Traceback (most recent call last): File "/home/nic/.pyenv/versions/3.6.4/lib/python3.6/asyncio/unix_events.py", line 91, in add_signal_handler signal.set_wakeup_fd(self._csock.fileno()) ValueError: set_wakeup_fd only works in main thread During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/nic/.pyenv/versions/3.6.4/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/home/nic/.pyenv/versions/3.6.4/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/nic/tmp/signal_event_loop_bug.py", line 14, in do_loop loop.add_signal_handler(signal.SIGINT, mysighandler) File "/home/nic/.pyenv/versions/3.6.4/lib/python3.6/asyncio/unix_events.py", line 93, in add_signal_handler raise RuntimeError(str(exc)) RuntimeError: set_wakeup_fd only works in main thread Code: import asyncio import signal import threading def mysighandler(): pass def do_loop(): loop = asyncio.new_event_loop() # This will fail with RuntimeError: set_wakeup_fd only works in main thread loop.add_signal_handler(signal.SIGINT, mysighandler) loop.close() t = threading.Thread(target=do_loop) t.start() t.join() |
|
|
| msg326426 - (view) |
Author: Andrew Svetlov (asvetlov) *  |
Date: 2018-09-26 08:01 |
| In my opinion, we should don't change the behavior. Signal handlers should be processed in main thread at the end, that's how Python works for regular synchronous API. Mentioning it in the doc makes sense. Would you make a pull request? |
|
|
| msg347525 - (view) |
Author: Brett Slatkin (haxor) |
Date: 2019-07-09 07:24 |
| I believe the scope of this bug may be larger than it originally seemed. Now that ProactorEventLoop is the default for Python 3.8 (https://bugs.python.org/issue34687), I'm seeing this same problem on Windows when you try to call asyncio.new_event_loop() from within a thread. It breaks with the ProactorEventLoop (snippet #1 below). It works fine with the SelectorEventLoop (snippet #2 below). Am I wrong to expect to be able to create a unique event loop for each thread from within the thread itself? I worked around this problem by manually forcing the event loop policy (https://bugs.python.org/issue33792). === Snippet #1 (breaks) === import asyncio from threading import Thread def my_func(): asyncio.new_event_loop() t = Thread(target=my_func) t.start() t.join() === Output from snippet #1 === PS G:\> python .\repro.py Exception in thread Thread-1: Traceback (most recent call last): File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner self.run() File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\threading.py", line 870, in run self._target(*self._args, **self._kwargs) File ".\repro.py", line 6, in my_func asyncio.new_event_loop() File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\events.py", line 758, in new_event_loop return get_event_loop_policy().new_event_loop() File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\events.py", line 656, in new_event_loop return self._loop_factory() File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\windows_events.py", line 310, in __init__ super().__init__(proactor) File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\proactor_events.py", line 630, in __init__ signal.set_wakeup_fd(self_no) ValueError: set_wakeup_fd only works in main thread === Snippet #2 (works) === import asyncio from threading import Thread # Work-around from https://bugs.python.org/issue34679 policy = asyncio.get_event_loop_policy() policy._loop_factory = asyncio.SelectorEventLoop def my_func(): asyncio.new_event_loop() t = Thread(target=my_func) t.start() t.join() === More details === My version of Python: 3.8.0b2 (tags/v3.8.0b2:21dd01d, Jul 4 2019, 16:00:09) [MSC v.1916 64 bit (AMD64)] My version of Windows (it's a developer VM from https://developer.microsoft.com/en-us/windows/downloads/virtual-machines): PS G:\> [System.Environment]::OSVersion.Version Major Minor Build Revision ----- ----- ----- -------- 10 0 17763 0 |
|
|
| msg347546 - (view) |
Author: Andrew Svetlov (asvetlov) * |
Date: 2019-07-09 11:08 |
| Good point. For proactor event loop set_wakeup_fd() is used for Ctrl+C handling. Skipping this call for non-main thread in proactor implementation makes sense. |
|
|
| msg350288 - (view) |
Author: Łukasz Langa (lukasz.langa) * |
Date: 2019-08-23 14:03 |
| Please fix this ASAP, last 3.8 beta is scheduled for Monday. |
|
|
| msg350298 - (view) |
Author: Yury Selivanov (yselivanov) * |
Date: 2019-08-23 14:32 |
| Andrew, can you fix ctrl-c/windows issue? I'm basically afk until monday. And we're not going to do anything about add_signal_handler in 3.8. |
|
|
| msg350304 - (view) |
Author: Brett Slatkin (haxor) |
Date: 2019-08-23 15:05 |
| Maybe we should just roll back https://bugs.python.org/issue34687 to fix both issues? Otherwise asyncio will be broken on Windows in 3.8. Is general API stability more important than performance? |
|
|
| msg350399 - (view) |
Author: Kyle Stanley (aeros) * |
Date: 2019-08-24 20:17 |
| > Skipping this call for non-main thread in proactor implementation makes sense. How do we identify whether or not set_wakeup_fd() is being called from a non-main thread? |
|
|
| msg350401 - (view) |
Author: Kyle Stanley (aeros) * |
Date: 2019-08-24 21:27 |
| > How do we identify whether or not set_wakeup_fd() is being called from a non-main thread? Never mind, I think I found the answer to my own question and tested a patch locally, I'll open a PR. |
|
|
| msg350463 - (view) |
Author: Andrew Svetlov (asvetlov) * |
Date: 2019-08-25 15:14 |
| The issue is related to Python 3.8 and master only. 3.6-3.7 are not affected |
|
|
| msg350464 - (view) |
Author: Andrew Svetlov (asvetlov) * |
Date: 2019-08-25 15:15 |
| Kyle, thanks for the fix. I have basically the same change in my PR but with test and news note. |
|
|
| msg350472 - (view) |
Author: Kyle Stanley (aeros) * |
Date: 2019-08-25 20:47 |
| > Kyle, thanks for the fix. > I have basically the same change in my PR but with test and news note. No problem, that works for me. I was mostly just trying to help with resolving some of the release blockers for 3.8b4. |
|
|
| msg350515 - (view) |
Author: Andrew Svetlov (asvetlov) * |
Date: 2019-08-26 09:51 |
| New changeset 1c0600998681295735a18690fae184b0c9a4ca51 by Andrew Svetlov in branch 'master': bpo-34679: Restore instantiation Windows IOCP event loop from non-main thread (#15492) https://github.com/python/cpython/commit/1c0600998681295735a18690fae184b0c9a4ca51 |
|
|
| msg350518 - (view) |
Author: miss-islington (miss-islington) |
Date: 2019-08-26 10:14 |
| New changeset 69d22b8fee442c12829a1032a72489c8133de271 by Miss Islington (bot) in branch '3.8': bpo-34679: Restore instantiation Windows IOCP event loop from non-main thread (GH-15492) https://github.com/python/cpython/commit/69d22b8fee442c12829a1032a72489c8133de271 |
|
|
| msg355230 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-10-23 15:25 |
| New changeset 1b53a24fb4417c764dd5933bce505f5c94249ca6 by Victor Stinner in branch 'master': bpo-34679: ProactorEventLoop only uses set_wakeup_fd() in main thread (GH-16901) https://github.com/python/cpython/commit/1b53a24fb4417c764dd5933bce505f5c94249ca6 |
|
|
| msg355233 - (view) |
Author: miss-islington (miss-islington) |
Date: 2019-10-23 15:44 |
| New changeset cbf474c98e702d12c97cd16a1e44ede10ea52b5b by Miss Skeleton (bot) in branch '3.8': bpo-34679: ProactorEventLoop only uses set_wakeup_fd() in main thread (GH-16901) https://github.com/python/cpython/commit/cbf474c98e702d12c97cd16a1e44ede10ea52b5b |
|
|