Issue 33097: concurrent futures Executors accept tasks after interpreter shutdown (original) (raw)

Created on 2018-03-18 16:41 by mrknmc, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 6144 merged python-dev,2018-03-18 16:42
PR 6445 merged miss-islington,2018-04-10 17:24
Messages (6)
msg314044 - (view) Author: Mark Nemec (mrknmc) * Date: 2018-03-18 16:41
Currently, one can submit a task to an executor (both ThreadPoolExecutor and ProcessPoolExecutor) during interpreter shutdown. One way to do this is to register function fun with atexit as below. @atexit.register def fun(): pool.submit(print, "apple") The future is accepted and goes into PENDING state. However, this can cause issues if the _python_exit function (located in concurrent/futures/thread.py and/or concurrent/futures/process.py) executes before function fun. Function _python_exit will shutdown the running workers in the pool and hence there will be no workers running by the time fun is executed so the future will be left in PENDING state forever. The solution submitted here is to instead raise a RuntimeException when a task is submitted during interpreter shutdown. This is the same behaviour as when the shutdown method of an executor is called explicitly.
msg314313 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-03-23 16:35
Thanks for spotting this. I will take a look soon, unless someone beats me to it.
msg315174 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-04-10 17:23
New changeset c4b695f85e141f57d22d8edf7bc2c756da136918 by Antoine Pitrou (Mark Nemec) in branch 'master': bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit (GH-6144) https://github.com/python/cpython/commit/c4b695f85e141f57d22d8edf7bc2c756da136918
msg315175 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-04-10 18:35
New changeset b26265900a18a184997c3c3a1fa6a5bf29703ec9 by Antoine Pitrou (Miss Islington (bot)) in branch '3.7': bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit (GH-6144) (GH-6445) https://github.com/python/cpython/commit/b26265900a18a184997c3c3a1fa6a5bf29703ec9
msg315176 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-04-10 18:38
Thanks your contribution Mark!
msg315182 - (view) Author: Mark Nemec (mrknmc) * Date: 2018-04-10 21:36
Happy to contribute! Thanks for the review :)
History
Date User Action Args
2022-04-11 14:58:58 admin set github: 77278
2018-04-10 21:36:03 mrknmc set messages: +
2018-04-10 18:38:59 pitrou set status: open -> closedversions: - Python 3.6messages: + resolution: fixedstage: patch review -> resolved
2018-04-10 18:35:11 pitrou set messages: +
2018-04-10 17:24:31 miss-islington set pull_requests: + <pull%5Frequest6140>
2018-04-10 17:23:16 pitrou set messages: +
2018-03-23 16:35:05 pitrou set nosy: + pitroumessages: +
2018-03-23 16:34:26 pitrou set nosy: + tomMoralversions: + Python 3.7, Python 3.8
2018-03-18 16:55:48 mrknmc set title: concurrent.futures executors accept tasks after interpreter shutdown -> concurrent futures Executors accept tasks after interpreter shutdown
2018-03-18 16:42:03 python-dev set keywords: + patchstage: patch reviewpull_requests: + <pull%5Frequest5901>
2018-03-18 16:41:13 mrknmc create