Issue 33082: multiprocessing docs bury very important 'callback=' args (original) (raw)

Created on 2018-03-15 18:50 by chadmiller-amzn, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 6122 closed chadmiller-amzn,2018-03-15 21:10
PR 6455 closed chadmiller-amzn,2018-04-12 01:05
Messages (9)
msg313905 - (view) Author: Chad (chadmiller-amzn) * Date: 2018-03-15 18:50
Callbacks are really important in multiprocessing. Doc writer almost ignores them.
msg313906 - (view) Author: Chad (chadmiller-amzn) * Date: 2018-03-15 18:55
https://github.com/chadmiller-amzn/cpython/pull/1 (Putting that in "GitHub PR" field says "Edit Error: Unknown PR format, acceptable formats are: full github URL, #pr_number, pr_number")
msg313915 - (view) Author: Mariatta (Mariatta) * (Python committer) Date: 2018-03-15 20:46
Can you make your PR in python/cpython GitHub repo, instead of your own fork?
msg314162 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-03-20 19:45
I don't really agree that "callbacks are really important in multiprocessing" (I think I've hardly ever used them). I agree with the other doc changes in the PR.
msg314267 - (view) Author: Chad (chadmiller-amzn) * Date: 2018-03-22 15:52
On topic: My CLA is signed as of Monday, 19 March. My status here is not updated yet. pitrou, off-topic: Without callbacks, users who want multiprocessing functions to return something, not just mutate state somewhere else, must gather jobs in a list and continually iterate through them polling to see if each has finished yet and conditionally popping it from the list. It's expensive and ugly and error-prone. Callbacks are really great, you should try them again. So much better: pool.apply_async(func, args, callback=when_finished_call_with_result)
msg319100 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-06-08 19:47
These doc changes have been languishing. We have a difference of opinion about adding a callback example. I don't have a strong feeling one way or the other. Davin or anyone else, what say you?
msg352366 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2019-09-13 16:55
I agree Antoine on this one, if one want the result, It'll get it from the returned value (.get method in the example), or simply by using the not-async versions and directly get the results as a return value of the call. Also the given example in the documentation won't work as the result is never waited for, the pool gets destroyed before the sleep have the time to sleep.
msg352367 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2019-09-13 16:55
I'd wait for Davin's review, but I'd keep the documentation as they are.
msg352375 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2019-09-13 17:16
I appreciate the functionality offered by the callbacks and have found good uses for them, as Chad clearly does/has. That said, the thought of expanding the documentation on the callbacks had not come up for me. Reading through the proposed changes to the prose explanations, the choice of words has changed but not significantly and virtually no new concepts are being explained. I agree with Julien that the docs should stay as they are. Chad: Thank you for advocating for things you think more people need to know about even if we do not update the docs this time.
History
Date User Action Args
2022-04-11 14:58:58 admin set github: 77263
2019-09-13 17:16:38 davin set status: open -> closedresolution: rejectedmessages: + stage: patch review -> resolved
2019-09-13 16:55:59 mdk set messages: +
2019-09-13 16:55:25 mdk set nosy: + mdkmessages: +
2018-06-08 19:47:47 ned.deily set nosy: + ned.deilymessages: +
2018-04-12 01:05:48 chadmiller-amzn set pull_requests: + <pull%5Frequest6150>
2018-03-22 15:52:55 chadmiller-amzn set messages: +
2018-03-20 19:45:29 pitrou set messages: +
2018-03-15 21:10:27 chadmiller-amzn set keywords: + patchstage: patch reviewpull_requests: + <pull%5Frequest5886>
2018-03-15 20:46:42 Mariatta set versions: - Python 3.4, Python 3.5
2018-03-15 20:46:10 Mariatta set nosy: + Mariattamessages: +
2018-03-15 18:55:08 chadmiller-amzn set messages: + versions: + Python 3.4, Python 3.5
2018-03-15 18:51:31 ned.deily set nosy: + pitrou, davinversions: - Python 3.4, Python 3.5
2018-03-15 18:50:50 chadmiller-amzn create