Issue 10437: ThreadPoolExecutor should accept max_workers=None (original) (raw)

This issue has been migrated to GitHub: https://github.com/python/cpython/issues/54646

classification

process

Created on 2010-11-16 19:36 by stutzbach, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (6)
msg121311 - (view)	Author: Daniel Stutzbach (stutzbach)	Date: 2010-11-16 19:36
ProcessPoolExecutor allows the max_workers parameter to the constructor to be omitted or None, in which case it will set the max_workers based on the number of CPUs. It would be nice if ThreadPoolExecutor's constructor worked the same way; having the same interface is one of the wonderful things about this pair of classes. I have a patch pretty much ready (with tests and a doc update), but it causes concurrent/futures/thread.py to import multiprocessing. Were you trying to avoid that?
msg121592 - (view)	Author: Brian Quinlan (bquinlan) *	Date: 2010-11-20 03:53
Daniel, I wasn't trying to avoid importing multiprocessing. What's your use case though? I think that defaulting the number of threads to the numbers of CPUs would trick users into believing that threads are useful for CPU-intensive work in Python ;-)
msg121709 - (view)	Author: Daniel Stutzbach (stutzbach)	Date: 2010-11-20 18:30
In my case, each thread launches a CPU-intensive process via subprocess, then waits for it to report output and exit. Also, keep in mind that PyPy and IronPython don't have a GIL (although I'm not a PyPy or IronPython user myself).
msg121718 - (view)	Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *	Date: 2010-11-20 18:58
pypy does have a GIL!
msg121866 - (view)	Author: Brian Quinlan (bquinlan) *	Date: 2010-11-21 05:38
I think that using the number of CPUs for max_workers makes sense for you but won't for most users. So I wouldn't make it a default.
msg122173 - (view)	Author: Daniel Stutzbach (stutzbach)	Date: 2010-11-22 22:35
Amaury Forgeot d'Arc said: > pypy does have a GIL! D'oh. That shows you how much I know about PyPy. Make that "keep in mind that IronPython doesn't have a GIL". ;) Brian Quinlan said: > I think that using the number of CPUs for max_workers makes sense > for you but won't for most users. So I wouldn't make it a default. Fair enough. Marking as invalid.