os.fork: documenting quirks and preventing it in debug mode · Issue #347 · python/asyncio (original) (raw)

This repository was archived by the owner on Nov 23, 2017. It is now read-only.

This repository was archived by the owner on Nov 23, 2017. It is now read-only.

Description

Long story short, one should never try to fork in a running event loop. Things that will break:

1. kqueue/epoll selectors will (likely) error with EBADF as soon as the forked child process does anything with them.
2. Some resources will leak.
3. Some resources will be closed incrorrectly, or, worst case, the queued data maybe be sent more than once.

Sadly, it's not possible to reliably fix the above. Hence this is a documentation issue, and maybe we can do something in debug mode.

Documentation

The only safe way of doing fork is to do it with no active event loop in the process. If it's absolutely needed, the semi-safe method of forking a running event loop is this:

1. Have a global variable fork_requested set to False
2. Before forking, fork_requested should be set to True, and loop.stop() method should be called.
3. The loop will stop, and the code will continue to execute after the loop.run_forever or loop.run_until_complete statement (however the loop was started).
4. At that place, you should check if fork_requested is True, and if it is, perform the fork. It's safe to call loop.close() in the child process, and to re-start the loop in the parent one.

Debug Mode

Monkey-patch os.fork to error out in debug mode when it's called. This isn't the prettiest solution, but it will work reliably. Moreover, in uvloop, for instance, it's either this, or a program crash (abort() called in C by libuv).