[Python-Dev] socketserver ForkingMixin waiting for child processes (original) (raw)

Victor Stinner victor.stinner at gmail.com
Wed Aug 16 10:13:09 EDT 2017

Hi,

The first bug was that test_socketserver "leaked" child processes: it means that socketserver API creates zombie processes depending how long the child processes take to complete.

If you want to backport my change waiting until child processes complete, you need to fix the bug differently, so test_socketserver doesn't leak processes (ex: wait until child processes complete in test_socketserver).

2017-08-12 0:34 GMT+02:00 Ryan Smith-Roberts <rmsr at lab.net>:

I agree that blocking shutdown by default isn't a good idea. A child will eventually get indefinitely stuck on a nonresponsive connection and hang the whole server. This behavior change is surprising and should be reverted in master, and definitely not backported.

Right, a child process can hang. The question is how to handle such case. I see 3 choices:

In Python 3.6, the process is stuck and continue to run even after shutdown() and server_close(). That's surprising and doesn't seem right to me.

As for block-timeout or block-timeout-kill, waiting more than zero seconds in serverclose() should be optional, because you're right that the best timeout is circumstantial.

Maybe we need to add a new method to wait N seconds until children completes, then send SIGTERM, and finally send SIGKILL if children take longer than N seconds to complete.

So the developer becomes responsible of killing child processes.

In the Apache world, it's called the "graceful" stop: https://httpd.apache.org/docs/2.4/stopping.html

Since ThreadingMixIn also leaks threads, serverclose() could grow a timeout flag (following the socket module timeout convention) and maybe a terminate boolean.

Oh, I didn't know that ThreadingMixIn also leaks threads.

That's a similar but different issue. It's not easily possible to "kill" a thread.

Plus it can't be backported to the feature-freeze branches.

IMHO leaking zombie processes is a bug, but I'm ok to keep bugs and only fix tests.

Victor

More information about the Python-Dev mailing list