Issue 33090: race condition between send and recv in _ssl with non-zero timeout (original) (raw)

Environment: Several versions of Python (see below), macOS 10.12.6

The attached script creates an SSL echo server (fairly standard), connects to the server, and spawns a read and write thread.

The write thread repeatedly shovels data into the connection, while the read thread receives data and prints a dot for each successful read.

The socket has a timeout of 10 seconds set: if the timeout is 0, the script blows up immediately due to blocking, and if the timeout is -1 nothing bad happens.

On Linux and the default Mac Python 2.6, the script prints an endless series of dots as expected.

On most other versions of Mac Python (2.7, 3.5, 3.6), the script dies quite quickly (within 1-2 seconds) with an error like this:

$ /usr/bin/python2.7 test_ssl.py Got connection from ('127.0.0.1', 49683) ..................................Exception in thread ReadThread: Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run self.__target(*self.__args, **self.__kwargs) File "test_ssl.py", line 93, in read_thread csocket.recv() File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 734, in recv return self.read(buflen) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 621, in read v = self._sslobj.read(len or 1024) error: [Errno 35] Resource temporarily unavailable

The error can be one of the following:

[Py2.7] error: [Errno 35] Resource temporarily unavailable [Py2.7] SSLWantReadError: The operation did not complete (read) (_ssl.c:1752) [Py3.x] BlockingIOError: [Errno 35] Resource temporarily unavailable [Py3.x] ssl.SSLWantReadError: The operation did not complete (read) (_ssl.c:1974) [Py3.6] ssl.SSLError: Invalid error code (_ssl.c:2217)

The last error occurs under much rarer circumstances, but appears to be associated with the same underlying bug. The "invalid error code" is 0 when tested with a debugger, indicating a successful completion (but somehow the error logic gets triggered anyway).

This was tested with the following configurations:

macOS: /usr/bin/python2.6: Python 2.6.9 from Apple [ok] macOS: /usr/bin/python2.7: Python 2.7.10 from Apple [crashes] macOS: /usr/local/bin/python2.7: Python 2.7.13 from Python.org [crashes] macOS: /usr/local/bin/python3.5: Python 3.5.1 from Python.org [crashes] macOS: /usr/local/bin/python3.6: Python 3.6.4 from Python.org [crashes] macOS: /opt/local/bin/python2.7: Python 2.7.14 from MacPorts [crashes]

A number of these were tested on a second machine (to rule out any strange environment issues), and the same results were obtained.

The script still fails on macOS 10.15 with python 3.9 (installed from python.org).

Interestingly enough:

$ python3.9 t.py Got connection from ('127.0.0.1', 60640) Exception in thread ReadThread: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 950, in _bootstrap_inner self.run() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 888, in run self._target(*self._args, **self._kwargs) File "/Users/ronald/Projects/python/github/cpython-ronald/t.py", line 93, in read_thread csocket.recv() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1226, in recv return self.read(buflen) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1101, in read return self._sslobj.read(len) BlockingIOError: [Errno 35] Resource temporarily unavailable

Got connection from ('127.0.0.1', 60651) .Traceback (most recent call last): File "/Users/ronald/Projects/python/github/cpython-ronald/t.py", line 74, in server_thread Exception in thread WriteThread: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 950, in _bootstrap_inner Exception in thread ReadThread: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 950, in _bootstrap_inner deal_with_client(newsocket) File "/Users/ronald/Projects/python/github/cpython-ronald/t.py", line 68, in deal_with_client data = connstream.read() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1101, in read self.run() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 888, in run self.run() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 888, in run self._target(*self._args, **self._kwargs) File "/Users/ronald/Projects/python/github/cpython-ronald/t.py", line 93, in read_thread self._target(*self._args, **self._kwargs) File "/Users/ronald/Projects/python/github/cpython-ronald/t.py", line 89, in write_thread return self._sslobj.read(len) csocket.recv() ssl.SSLError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:2621) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1226, in recv csocket.send(b'blah') Closed connection from ('127.0.0.1', 60651) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1173, in send return self.read(buflen) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1101, in read return self._sslobj.write(data) ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:2471) return self._sslobj.read(len) ssl.SSLError: [SSL: SSLV3_ALERT_BAD_RECORD_MAC] sslv3 alert bad record mac (_ssl.c:2621)

Got connection from ('127.0.0.1', 60636) .Exception in thread ReadThread: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 950, in _bootstrap_inner Traceback (most recent call last): File "/Users/ronald/Projects/python/github/cpython-ronald/t.py", line 74, in server_thread self.run() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 888, in run deal_with_client(newsocket) File "/Users/ronald/Projects/python/github/cpython-ronald/t.py", line 68, in deal_with_client self._target(*self._args, **self._kwargs) File "/Users/ronald/Projects/python/github/cpython-ronald/t.py", line 93, in read_thread csocket.recv() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1226, in recv data = connstream.read() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1101, in read return self.read(buflen) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1101, in read return self._sslobj.read(len) return self._sslobj.read(len) ssl.SSLError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:2621) Closed connection from ('127.0.0.1', 60636) BlockingIOError: [Errno 35] Resource temporarily unavailable Exception in thread WriteThread: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 950, in _bootstrap_inner