[Python-Dev] Help preventing SIGPIPE/SIG_DFL anti-pattern. (original) (raw)

Alfred Perlstein alfred at freebsd.org
Sat Jun 30 12:31:35 EDT 2018

(sorry for the double post, looks like maybe attachments are dropped, inlined the attachment this time.)

Hello,

I'm looking for someone in the python community to help with a problem of anti-patterns showing up dealing with SIGPIPE.

Specifically I've noticed an anti-pattern developing where folks will try to suppress broken pipe errors written to stdout by setting SIGPIPE's disposition to SIG_DFL.  This is actually very common, and also rather broken due to the fact that for all but the most simple text filters this opens up a problem where the process can exit unexpectedly due to SIGPIPE being generated from a remote connection the program makes.

I have attached a test program which shows the problem.

to use this program it takes several args.

1. Illustrate the 'ugly output to stderr' that folks want to avoid:

% python3 t0.py nocatch | head -1

2. Illustrate the anti-pattern, the program exits on about line 47

which most folks to not understand

% python3 t0.py dfl | head -1

3. Show a better solution where we catch the pipe error and cleanup to

avoid the message:

% python3 t0.py | head -1

I did a recent audit of a few code bases and saw this pattern pop often often enough that I am asking if there's a way we can discourage the use of "signal(SIGPIPE, SIG_DFL)" unless the user really understands what they are doing.

I do have a pull req here: https://github.com/python/cpython/pull/6773 where I am trying to document this on the signal page, but I can't sort out how to land this doc change.

thank you,

-Alfred

=== CUT HERE ===

Program showing the dangers of setting the SIG_PIPE handler to the

default handler (SIG_DFL). #

To illustrate the problem run with:

./foo.py dfl

The program will exit in do_network_stuff() even though there is a an

"except" clause.

The do_network_stuff() simulates a remote connection that closes

before it can be written to

which happens often enough to be a hazard in practice.

import signal import sys import socket import os

def sigpipe_handler(sig, frame):     sys.stderr.write("Got sigpipe \n\n\n")     sys.stderr.flush()

def get_server_connection():     # simulate making a connection to a remote service that closes the connection     # before we can write to it.  (In practice a host rebooting, or otherwise exiting while we are     # trying to interact with it will be the true source of such behavior.)     s1, s2 = socket.socketpair()     s2.close()     return s1

def do_network_stuff():     # simulate interacting with a remote service that closes its connection     # before we can write to it.  Example: connecting to an http service and     # issuing a GET request, but the remote server is shutting down between     # when our connection finishes the 3-way handshake and when we are able     # to write our "GET /" request to it.     # In theory this function should be resilient to this, however if SIGPIPE is set     # to SIGDFL then this code will cause termination of the program.     if 'dfl' in sys.argv[1:]:         signal.signal(signal.SIGPIPE, signal.SIG_DFL)

    for x in range(5):         server_conn = get_server_connection()         sys.stderr.write("about to write to server socket...\n")         try:             server_conn.send(b"GET /")         except BrokenPipeError as bpe:             sys.stderr.write("caught broken pipe on talking to server, retrying...")

def work():     do_network_stuff()     for x in range(10000):         print("y")     sys.stdout.flush()

def main():     if 'nocatch' in sys.argv[1:]:         work()     else:         try:             work()         except BrokenPipeError as bpe:             signal.signal(signal.SIGPIPE, signal.SIG_DFL)             os.kill(os.getpid(), signal.SIGPIPE)

if name == 'main':     main()

More information about the Python-Dev mailing list