[Python-Dev] A more flexible task creation (original) (raw)

Steve Holden steve at holdenweb.com
Fri Jun 15 03:56:49 EDT 2018

On Thu, Jun 14, 2018 at 8:14 PM, Chris Barker via Python-Dev < python-dev at python.org> wrote:

Excuse my ignorance (or maybe it's a vocabulary thing), but I'm trying to understand the problem here.

So why do queries fail with 10000 tasks? or ANY number? If the async DB access code is written right, a given query should not "await" unless it is in a safe state to do so.

So what am I missing here??? because threads ARE concurrent, and there is no advantage to having more threads than can actually run at once, and having many more does cause thread-switching performance issues.

To me, tasks are (somewhat) logically analogous to threads. kinda -- in the sense that they are run (and completed) in arbitrary order, But they are different, and that difference is key to this issue. As Yury expressed interest in this idea, there must be something I'm missing. What is it?

All tasks need resources, and bookkeeping for such tasks is likely to slow things down. More importantly, with an uncontrolled number of tasks you can require an uncontrolled use of resources, decreasing efficiency to levels well below that attainable with sensible conservation of resources. Imagine, if you will, a task that starts by allocating 1GB of memory. Would you want 10,000 of those? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20180615/1acfed1d/attachment.html>

More information about the Python-Dev mailing list