Issue 31249: test_concurrent_futures leaks dangling threads (original) (raw)

Issue31249

Created on 2017-08-21 16:04 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 3167 merged vstinner,2017-08-21 16:12
PR 3178 merged vstinner,2017-08-22 10:41
PR 3253 merged vstinner,2017-08-31 13:54
PR 3521 merged vstinner,2017-09-12 23:49
PR 3522 merged python-dev,2017-09-13 00:06
Messages (8)
msg300634 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-08-21 16:04
Using PR 3138 of bpo-31234, I noticed that test_concurrent_futures leaks dangling threads: haypo@selma$ ./python -u -m test --fail-env-changed test_concurrent_futures -v -m ThreadPoolWaitTests == CPython 3.7.0a0 (heads/master:0267128, Aug 21 2017, 17:23:11) [GCC 6.4.1 20170727 (Red Hat 6.4.1-1)] (...) test_all_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... Warning -- threading_cleanup() detected 0 leaked threads (count: 0, dangling: 6) 0.11s ok test_first_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.60s ok test_first_completed_some_already_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.60s ok test_first_exception (test.test_concurrent_futures.ThreadPoolWaitTests) ... Warning -- threading_cleanup() detected 0 leaked threads (count: 0, dangling: 6) 3.11s ok test_first_exception_one_already_failed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 2.10s ok test_first_exception_some_already_complete (test.test_concurrent_futures.ThreadPoolWaitTests) ... Warning -- threading_cleanup() detected 0 leaked threads (count: 0, dangling: 6) 1.60s ok test_pending_calls_race (test.test_concurrent_futures.ThreadPoolWaitTests) ... 0.12s ok test_timeout (test.test_concurrent_futures.ThreadPoolWaitTests) ... 6.11s ok (...)
msg300651 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-08-21 21:24
New changeset 489d91c61d02f401551966f1558ee2dc87fc0ad9 by Victor Stinner in branch 'master': bpo-31249: test_concurrent_futures checks dangling threads (#3167) https://github.com/python/cpython/commit/489d91c61d02f401551966f1558ee2dc87fc0ad9
msg300697 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-08-22 14:50
New changeset bc61315377056fe362b744d9c44e17cd3178ce54 by Victor Stinner in branch 'master': bpo-31249: Fix ref cycle in ThreadPoolExecutor (#3178) https://github.com/python/cpython/commit/bc61315377056fe362b744d9c44e17cd3178ce54
msg301098 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-01 12:46
New changeset 60f3f1fb5cbf5354c3081138be806c56cb04153f by Victor Stinner in branch '3.6': bpo-31249: Fix ref cycle in ThreadPoolExecutor (#3253) https://github.com/python/cpython/commit/60f3f1fb5cbf5354c3081138be806c56cb04153f
msg302007 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-12 23:33
I still see the warning sometimes: 0:04:09 load avg: 7.03 [185/405/1] test_concurrent_futures failed (env changed) (68 sec) -- running: test_multiprocessing_spawn (160 sec), test_gdb (53 sec), test_lib2to3 (195 sec) test_cancel (test.test_concurrent_futures.FutureTests) ... ok test_cancelled (test.test_concurrent_futures.FutureTests) ... ok test_done (test.test_concurrent_futures.FutureTests) ... ok test_done_callback_already_cancelled (test.test_concurrent_futures.FutureTests) ... ok test_done_callback_already_failed (test.test_concurrent_futures.FutureTests) ... ok test_done_callback_already_successful (test.test_concurrent_futures.FutureTests) ... ok test_done_callback_raises (test.test_concurrent_futures.FutureTests) ... ok test_done_callback_with_cancel (test.test_concurrent_futures.FutureTests) ... ok test_done_callback_with_exception (test.test_concurrent_futures.FutureTests) ... ok test_done_callback_with_result (test.test_concurrent_futures.FutureTests) ... ok test_exception_with_success (test.test_concurrent_futures.FutureTests) ... ok test_exception_with_timeout (test.test_concurrent_futures.FutureTests) ... ok test_repr (test.test_concurrent_futures.FutureTests) ... ok test_result_with_cancel (test.test_concurrent_futures.FutureTests) ... ok test_result_with_success (test.test_concurrent_futures.FutureTests) ... ok test_result_with_timeout (test.test_concurrent_futures.FutureTests) ... ok test_running (test.test_concurrent_futures.FutureTests) ... ok test_correct_timeout_exception_msg (test.test_concurrent_futures.ProcessPoolAsCompletedTests) ... 0.15s ok test_duplicate_futures (test.test_concurrent_futures.ProcessPoolAsCompletedTests) ... 2.16s ok test_free_reference_yielded_future (test.test_concurrent_futures.ProcessPoolAsCompletedTests) ... 0.15s ok test_no_timeout (test.test_concurrent_futures.ProcessPoolAsCompletedTests) ... 0.15s ok test_zero_timeout (test.test_concurrent_futures.ProcessPoolAsCompletedTests) ... 2.15s ok test_free_reference (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.17s ok test_killed_child (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.15s ok test_map (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.18s ok test_map_chunksize (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.17s ok test_map_exception (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.17s ok test_map_timeout (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 6.16s ok test_max_workers_negative (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.14s ok test_no_stale_references (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.14s ok test_shutdown_race_issue12456 (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.15s ok test_submit (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.14s ok test_submit_keyword (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.13s ok test_traceback (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 0.15s ok test_context_manager_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 0.07s ok test_del_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 0.05s Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2) ok test_hang_issue12364 (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 1.09s ok test_interpreter_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 2.67s ok test_processes_terminate (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 0.05s ok test_run_after_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 0.00s ok test_all_completed (test.test_concurrent_futures.ProcessPoolWaitTests) ... 0.16s ok test_first_completed (test.test_concurrent_futures.ProcessPoolWaitTests) ... 1.64s ok test_first_completed_some_already_completed (test.test_concurrent_futures.ProcessPoolWaitTests) ... 1.65s ok test_first_exception (test.test_concurrent_futures.ProcessPoolWaitTests) ... 3.15s ok test_first_exception_one_already_failed (test.test_concurrent_futures.ProcessPoolWaitTests) ... 2.15s ok test_first_exception_some_already_complete (test.test_concurrent_futures.ProcessPoolWaitTests) ... 1.65s ok test_timeout (test.test_concurrent_futures.ProcessPoolWaitTests) ... 6.15s ok test_correct_timeout_exception_msg (test.test_concurrent_futures.ThreadPoolAsCompletedTests) ... 0.16s ok test_duplicate_futures (test.test_concurrent_futures.ThreadPoolAsCompletedTests) ... 2.15s ok test_free_reference_yielded_future (test.test_concurrent_futures.ThreadPoolAsCompletedTests) ... 0.15s ok test_no_timeout (test.test_concurrent_futures.ThreadPoolAsCompletedTests) ... 0.16s ok test_zero_timeout (test.test_concurrent_futures.ThreadPoolAsCompletedTests) ... 2.15s ok test_default_workers (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.15s ok test_free_reference (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.14s ok test_map (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.15s ok test_map_exception (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.16s ok test_map_submits_without_iteration (test.test_concurrent_futures.ThreadPoolExecutorTest) Tests verifying issue 11777. ... 0.15s ok test_map_timeout (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 6.13s ok test_max_workers_negative (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.14s ok test_no_stale_references (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.14s ok test_shutdown_race_issue12456 (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.16s ok test_submit (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.14s ok test_submit_keyword (test.test_concurrent_futures.ThreadPoolExecutorTest) ... 0.14s ok test_context_manager_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.07s ok test_del_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.04s ok test_hang_issue12364 (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 1.07s ok test_interpreter_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 1.65s ok test_run_after_shutdown (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.00s ok test_thread_names_assigned (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.05s ok test_thread_names_default (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.04s ok test_threads_terminate (test.test_concurrent_futures.ThreadPoolShutdownTest) ... 0.02s ok test_all_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 0.14s ok test_first_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.65s ok test_first_completed_some_already_completed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.65s ok test_first_exception (test.test_concurrent_futures.ThreadPoolWaitTests) ... 3.14s ok test_first_exception_one_already_failed (test.test_concurrent_futures.ThreadPoolWaitTests) ... 2.16s ok test_first_exception_some_already_complete (test.test_concurrent_futures.ThreadPoolWaitTests) ... 1.62s ok test_pending_calls_race (test.test_concurrent_futures.ThreadPoolWaitTests) ... 0.16s ok test_timeout (test.test_concurrent_futures.ThreadPoolWaitTests) ... 6.13s ok ---------------------------------------------------------------------- Ran 79 tests in 68.182s OK
msg302011 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-13 00:05
New changeset 3bcf157c115ba3e48bce62ac8cb13c703475a113 by Victor Stinner in branch 'master': bpo-31249: Fix test_concurrent_futures dangling thread (#3521) https://github.com/python/cpython/commit/3bcf157c115ba3e48bce62ac8cb13c703475a113
msg302016 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-13 00:43
New changeset 94eb2d5b73d07bc447f1fe19923b4c3259734b9c by Victor Stinner (Miss Islington (bot)) in branch '3.6': [3.6] bpo-31249: Fix test_concurrent_futures dangling thread (GH-3521) (#3522) https://github.com/python/cpython/commit/94eb2d5b73d07bc447f1fe19923b4c3259734b9c
msg302017 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-13 00:45
Ok, the issue should now be fixed in Python 3.6 and master (future 3.7).
History
Date User Action Args
2022-04-11 14:58:51 admin set github: 75432
2017-09-13 00:45:29 vstinner set status: open -> closedresolution: fixedmessages: + stage: patch review -> resolved
2017-09-13 00:43:47 vstinner set messages: +
2017-09-13 00:06:16 python-dev set pull_requests: + <pull%5Frequest3520>
2017-09-13 00:05:55 vstinner set messages: +
2017-09-12 23:49:26 vstinner set keywords: + patchstage: resolved -> patch reviewpull_requests: + <pull%5Frequest3519>
2017-09-12 23:33:45 vstinner set status: closed -> openresolution: fixed -> (no value)messages: +
2017-09-01 12:49:12 vstinner set status: open -> closedresolution: fixedstage: resolved
2017-09-01 12:46:46 vstinner set messages: +
2017-08-31 13:54:53 vstinner set pull_requests: + <pull%5Frequest3297>
2017-08-22 14:50:45 vstinner set messages: +
2017-08-22 13:58:19 vstinner set versions: + Python 3.6
2017-08-22 10:41:00 vstinner set pull_requests: + <pull%5Frequest3216>
2017-08-21 21:24:26 vstinner set messages: +
2017-08-21 16:12:36 vstinner set pull_requests: + <pull%5Frequest3203>
2017-08-21 16:04:21 vstinner set type: resource usagecomponents: + Tests
2017-08-21 16:04:13 vstinner create