[Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7 (original) (raw)

Jim Baker jim.baker at python.org
Wed Jun 7 15:06:39 EDT 2017

On Wed, Jun 7, 2017 at 2:31 AM, Cory Benfield <cory at lukasa.co.uk> wrote:

On 6 Jun 2017, at 18:49, Jim Baker <jim.baker at python.org> wrote: With Nick's suggestion of tlsbootstrap, this has certainly become more complicated. I'm still thinking of the ramifications for future Jython 2.7 support, if Python dev goes down this path. It still seems easier to me to avoid exposing the SSLObject/MemoryBIO model to 2.7 and instead concentrate on just having native TLS support. Yes, this might require more work than a simple backport, but everyone is resource constrained, not just the Requests or Jython teams.

Oh. This hadn’t occurred to me until just now, but doesn’t PyOpenSSL (or anything built on CFFI) just basically not run on Jython? Or am I mistaken?

Sorry, I wish this were true, but PyOpenSSL is not available on Jython, because we do not yet support CFFI for Jython. CFFI support is something we have looked at, but we have not implemented. (There is a related and far more ambitious project to implement support for the C Extension API, http://jyni.org/, which Stefan Richthofer is working on with me under the Google Summer of Code.)

Having said that, bundling PyOpenSSL for use by pip is something that we would not want to do for Jython itself. We want to use the native Java ecosystem where possible for built-in functionality, in part because we already have native support already to the underlying system trust store on all platforms. (Java development did all the heavy lifting for us.)

Instead our current implementation of socket/ssl/select is built on Netty, plus some additional work for managing SSLContext in such a way that is compatible with Python. There is an out-of-date doc I prepared describing what was done (but broad aspects are still current): https://github.com/ jimbaker/socket-reboot

My concrete suggestion is that any work on PEP 543 will benefit from improving the testing currently found in testssl as part of its implementation. For a variety of reasons, test functions like sslioloop ( https://github.com/python/cpython/blob/master/Lib/test/testssl.py#L1691) avoid asserting more than it can properly manage the framing of wrapped/unwrapped data, so for example one doesn't want to state explicitly when SSLERRORWANTREAD would be called (too much white box at that point). On the other hand, the lack of unit testing of, for example, SSLObject.read, but instead only doing at the functional test level, means that there's nothing explicit testing "will raise an SSLWantReadError if it needs more data than the incoming BIO has available" (per the docs), or similar use of signaling (SSLEOFError comes to mind).

Yeah, PEP 543 just basically assumes the stdlib’s testing of TLS doesn’t exist and that it’ll have to manufacture its own. Unfortunately, because it is attempting to abstract across many different implementations the tests will need to be fairly high-level: for example, there is no consistent way to discuss the actual size of the buffers in the buffer-based TLS implementation, and they are allowed to be unbounded buffers, so tests cannot validate that TLSWantReadError and TLSWantWriteError are ever actually raised: all they can do is run tests that will handle them in the even that they are raised and confirm the data is transferred appropriately.

Right, so such improved testing, regardless of level, will still be an improvement, and we look forward to seeing it for use by Jython as well.

I should mention that we sometimes see undocumented functionality leak out. For example, https://docs.python.org/3/library/socket.html#socket.socket.listen doesn't say anything about backlog=0, but the requests test suite (last time I looked on Jython) now depends on it. We assumed it was something like http://pubs.opengroup.org/onlinepubs/009695399/functions/listen.html, but as described in http://bugs.python.org/issue8498, it now means that "a server application in python that accepts exactly 1 connection", by passing to the underlying C. More tests, more docs, please, even though of course that's a lot of dev effort.

More information about the Python-Dev mailing list