Issue 31250: test_asyncio leaks dangling threads (original) (raw)
Created on 2017-08-21 16:33 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.
Messages (7)
Author: STINNER Victor (vstinner) * 
Date: 2017-08-21 16:33
Using PR 3138 of bpo-31234, I noticed that test_asyncio leaks dangling threads.
Tests which randomly leak threads:
test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_no_inet_pton test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_service_name test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_bluetooth test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_datagram_endpoint_no_addrinfo test.test_asyncio.test_futures.DuckTests.test_wrap_future test.test_asyncio.test_futures.CFutureTests.test_wrap_future test.test_asyncio.test_futures.CFutureTests.test_wrap_future_use_global_loop test.test_asyncio.test_futures.PyFutureTests.test_wrap_future test.test_asyncio.test_futures.PyFutureTests.test_wrap_future_use_global_loop test.test_asyncio.test_selector_events.BaseSelectorEventLoopTests.test_sock_connect_resolve_using_socket_params test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_with_timeout test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_task_cancelled test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_task_factory_exception
Write this list into "tests" and run "python3 -m test --matchfile=tests".
haypo@selma$ ./python -u -m test -v test_asyncio --matchfile=tests == CPython 3.7.0a0 (heads/concurrent_futures:f277fa3, Aug 21 2017, 18:11:41) [GCC 6.4.1 20170727 (Red Hat 6.4.1-1)] (...) test_create_connection_bluetooth (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... ok test_create_connection_no_inet_pton (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 4 leaked threads (count: 4, dangling: 5) ok test_create_connection_service_name (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 4 leaked threads (count: 4, dangling: 5) ok test_create_datagram_endpoint_no_addrinfo (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2) ok test_wrap_future (test.test_asyncio.test_futures.CFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2) ok test_wrap_future_use_global_loop (test.test_asyncio.test_futures.CFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2) ok test_wrap_future (test.test_asyncio.test_futures.DuckTests) ... ok test_wrap_future (test.test_asyncio.test_futures.PyFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2) ok test_wrap_future_use_global_loop (test.test_asyncio.test_futures.PyFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2) ok test_sock_connect_resolve_using_socket_params (test.test_asyncio.test_selector_events.BaseSelectorEventLoopTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2) ok test_run_coroutine_threadsafe (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests) Test coroutine submission from a thread to an event loop. ... ok test_run_coroutine_threadsafe_task_cancelled (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests) Test coroutine submission from a tread to an event loop ... ok test_run_coroutine_threadsafe_task_factory_exception (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests) Test coroutine submission from a tread to an event loop ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2) ok test_run_coroutine_threadsafe_with_timeout (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests) Test coroutine submission from a thread to an event loop ... ok (...)
Author: STINNER Victor (vstinner) *
Date: 2017-08-31 13:42
I proposed PR 3252 to only fix unit tests, but not asyncio.
An alternative is to modify BaseEventLoop.close() to replace executor.shutdown(wait=False) with executor.shutdown(wait=True), but it would change the behaviour. I'm not sure if we want to wait until all threads complete on close()?
Author: STINNER Victor (vstinner) *
Date: 2017-09-01 12:46
New changeset 16432beadb8eba079c9786cc0c0eaacfd9fd2f7b by Victor Stinner in branch 'master': bpo-31250, test_asyncio: fix dangling threads (#3252) https://github.com/python/cpython/commit/16432beadb8eba079c9786cc0c0eaacfd9fd2f7b
Author: STINNER Victor (vstinner) *
Date: 2017-09-01 12:52
I picked the safe option: only modify unit tests.
Author: STINNER Victor (vstinner) *
Date: 2017-09-01 15:55
Hum, EventLoopTestsMixin.tearDown() still has a bug. I reopen the issue.
Author: STINNER Victor (vstinner) *
Date: 2017-09-01 22:25
New changeset e8a533fbc734af6eeb389202ba6c6e9c2548027f by Victor Stinner in branch 'master': bpo-31250, test_asyncio: fix EventLoopTestsMixin.tearDown() (#3264) https://github.com/python/cpython/commit/e8a533fbc734af6eeb389202ba6c6e9c2548027f
Author: STINNER Victor (vstinner) *
Date: 2017-09-12 21:18
New changeset 5013a5ebc9978a58435036fa3860c465882c21da by Victor Stinner in branch '3.6': [3.6] bpo-31250: test_asyncio: fix dangling threads (#3517) https://github.com/python/cpython/commit/5013a5ebc9978a58435036fa3860c465882c21da
History
Date
User
Action
Args
2022-04-11 14:58:51
admin
set
github: 75433
2017-09-13 09:56:51
vstinner
set
versions: + Python 3.6
2017-09-12 21🔞39
vstinner
set
status: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-09-12 21🔞25
vstinner
set
messages: +
2017-09-12 17:30:33
vstinner
set
keywords: + patch
stage: resolved -> patch review
pull_requests: + <pull%5Frequest3512>
2017-09-01 22:25:42
vstinner
set
messages: +
2017-09-01 15:55:46
vstinner
set
pull_requests: + <pull%5Frequest3308>
2017-09-01 15:55:41
vstinner
set
status: closed -> open
resolution: fixed -> (no value)
messages: +
2017-09-01 12:52:46
vstinner
set
status: open -> closed
resolution: fixed
messages: +
stage: resolved
2017-09-01 12:46:09
vstinner
set
messages: +
2017-08-31 13:42:00
vstinner
set
messages: +
2017-08-31 13:39:56
vstinner
set
pull_requests: + <pull%5Frequest3296>
2017-08-21 16:33:56
vstinner
create