| msg186372 - (view) |
Author: Hiroaki Kawai (Hiroaki.Kawai) |
Date: 2013-04-09 04:45 |
| When using ssl module, I sometimes get unexpected error. The observed error varies in different situations. After the investigation, I found the reason was that ssl shutdown was not performed and sometimes RST was sent over the network instead of FIN. I created a patch against 2.7 branch. |
|
|
| msg186399 - (view) |
Author: Antoine Pitrou (pitrou) *  |
Date: 2013-04-09 13:15 |
| I don't think your patch is right: - calling unwrap() already shuts down the SSL layer; this is the right way to do it and is documented as such: "Performs the SSL shutdown handshake, which removes the TLS layer from the underlying socket, and returns the underlying socket object" - shutdown() right now isn't blocking; if you add a call to SSL shutdown, it can either block or fail with EAGAIN or similar, which is something people won't expect - close() should simply close the file descriptor, like on a regular socket (if you call socket.close(), it won't shutdown the TCP connection, especially if there's another file descriptor referencing the same connection) As for Modules/_ssl.c, the case where SSL_shutdown() returns 0 is already handled: if (err == 0) { /* Don't loop endlessly; instead preserve legacy behaviour of trying SSL_shutdown() only twice. This looks necessary for OpenSSL < 0.9.8m */ if (++zeros > 1) break; /* Shutdown was sent, now try receiving */ self->shutdown_seen_zero = 1; continue; } ... so I don't think anything more is necessary. So I think things are fine right now and your patch shouldn't be applied. |
|
|
| msg186415 - (view) |
Author: Hiroaki Kawai (Hiroaki.Kawai) |
Date: 2013-04-09 14:36 |
| Please run the test so that you'll see the problem. 2013/4/9 Antoine Pitrou <report@bugs.python.org> > > Antoine Pitrou added the comment: > > I don't think your patch is right: > > - calling unwrap() already shuts down the SSL layer; this is the right way > to do it and is documented as such: "Performs the SSL shutdown handshake, > which removes the TLS layer from the underlying socket, and returns the > underlying socket object" > > - shutdown() right now isn't blocking; if you add a call to SSL shutdown, > it can either block or fail with EAGAIN or similar, which is something > people won't expect > > - close() should simply close the file descriptor, like on a regular > socket (if you call socket.close(), it won't shutdown the TCP connection, > especially if there's another file descriptor referencing the same > connection) > > As for Modules/_ssl.c, the case where SSL_shutdown() returns 0 is already > handled: > > if (err == 0) { > /* Don't loop endlessly; instead preserve legacy > behaviour of trying SSL_shutdown() only twice. > This looks necessary for OpenSSL < 0.9.8m */ > if (++zeros > 1) > break; > /* Shutdown was sent, now try receiving */ > self->shutdown_seen_zero = 1; > continue; > } > > ... so I don't think anything more is necessary. > > So I think things are fine right now and your patch shouldn't be applied. > > ---------- > nosy: +pitrou > stage: -> patch review > versions: -Python 2.6, Python 3.1, Python 3.2, Python 3.5 > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17672> > _______________________________________ > |
|
|
| msg186417 - (view) |
Author: Charles-François Natali (neologix) * |
Date: 2013-04-09 14:45 |
| > sometimes RST was sent over the network instead of FIN Your client sends data, but the server never reads it: when a TCP socket is closed while there's still data in the input socket buffer, a RST is sent instead of a FIN. That's normal behaviour. |
|
|
| msg186418 - (view) |
Author: Hiroaki Kawai (Hiroaki.Kawai) |
Date: 2013-04-09 14:53 |
| Client gets an exception in reading the socket, not in writing. Please run the test code and see what happens. 2013/4/9 Charles-François Natali <report@bugs.python.org> > > Charles-François Natali added the comment: > > > sometimes RST was sent over the network instead of FIN > > Your client sends data, but the server never reads it: when a TCP socket > is closed while there's still data in the input socket buffer, a RST is > sent instead of a FIN. That's normal behaviour. > > ---------- > nosy: +neologix > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17672> > _______________________________________ > |
|
|
| msg186420 - (view) |
Author: Charles-François Natali (neologix) * |
Date: 2013-04-09 14:58 |
| > Client gets an exception in reading the socket, not in writing. Please run > the test code and see what happens. Of course it gets ECONNRESET on subsequent recv(), that's how TCP works. Just make your handler read from the socket and it won't happen anymore. |
|
|
| msg186424 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2013-04-09 15:05 |
| > Client gets an exception in reading the socket, not in writing. Yes, it does, and the exception bears the error code SSL_ERROR_EOF (8), which is expected here. The question is: why would you expect reading *not* to raise an exception while the remote end of the connection has been closed? TCPStreamHandler will only keep the connection alive as long as the handle() method is running, the connection is disposed of afterwards. |
|
|
| msg186425 - (view) |
Author: Hiroaki Kawai (Hiroaki.Kawai) |
Date: 2013-04-09 15:05 |
| As an interface of ssl socket, server does not have to read, just write some data. The client side should be able to read the bytes that ther server sent. The problem is that client will sometimes raise an unexpected SSLError in reading the ssl socket because server side does not shutdown the ssl session cleanly. 2013/4/9 Charles-François Natali <report@bugs.python.org> > > Charles-François Natali added the comment: > > > Client gets an exception in reading the socket, not in writing. Please > run > > the test code and see what happens. > > Of course it gets ECONNRESET on subsequent recv(), that's how TCP works. > > Just make your handler read from the socket and it won't happen anymore. > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17672> > _______________________________________ > |
|
|
| msg186426 - (view) |
Author: Charles-François Natali (neologix) * |
Date: 2013-04-09 15:10 |
| > As an interface of ssl socket, server does not have to read, just write > some data. > The client side should be able to read the bytes that ther server sent. The > problem is that client will sometimes raise an unexpected SSLError in > reading the ssl socket because server side does not shutdown the ssl > session cleanly. Once again, that's not how TCP works. If your server doesn't read data that the client sent it, a RST will be sent when the server closes its end. Please do read the TCP spec, or use google: http://cs.baylor.edu/~donahoo/practical/CSockets/TCPRST.pdf I suggest closing this issue... |
|
|
| msg186427 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2013-04-09 15:13 |
| > As an interface of ssl socket, server does not have to read, just > write > some data. > The client side should be able to read the bytes that ther server > sent. Please re-read your own code. The server does: + def handle(self): + self.wfile.write("123") and the client does: + s.recv(3) So, yes, the client is able to read the 3 bytes that the server sent. It's only when you are trying to read an additional byte that an exception is raised, because the TCP connection has been closed by the server. |
|
|
| msg186428 - (view) |
Author: Hiroaki Kawai (Hiroaki.Kawai) |
Date: 2013-04-09 15:20 |
| The error looks like : SSLError(8, '_ssl.c:1363: EOF occurred in violation of protocol') But why we see "in violation of protocol" here? 2013/4/10 Antoine Pitrou <report@bugs.python.org> > > Antoine Pitrou added the comment: > > > Client gets an exception in reading the socket, not in writing. > > Yes, it does, and the exception bears the error code SSL_ERROR_EOF (8), > which is expected here. > > The question is: why would you expect reading *not* to raise an exception > while the remote end of the connection has been closed? TCPStreamHandler > will only keep the connection alive as long as the handle() method is > running, the connection is disposed of afterwards. > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17672> > _______________________________________ > |
|
|
| msg186434 - (view) |
Author: Hiroaki Kawai (Hiroaki.Kawai) |
Date: 2013-04-09 15:44 |
| Ah, sorry I understood now. |
|
|
| msg186435 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2013-04-09 15:44 |
| > The error looks like : SSLError(8, '_ssl.c:1363: EOF occurred in > violation > of protocol') > But why we see "in violation of protocol" here? Because the SSL layer wasn't shutdown cleanly: the TCP connection was closed while the SSL layer was still active. You have three solutions around this: - you can call unwrap() for a clean SSL shutdown (the server has to call unwrap() too). - you can use suppress_ragged_eofs=True with wrap_socket() - you can simply avoid reading past the server's data, which will solve the problem altogether |
|
|
| msg186437 - (view) |
Author: Hiroaki Kawai (Hiroaki.Kawai) |
Date: 2013-04-09 16:03 |
| I think creating an ssl socket from existing socket from an instance generated by library routine, and replace that socket with ssl socket is very common usage. Injecting wrap_socket is very easy. But injecting unwrap call is not easy. In python 2.6, I got a plain socket.error of "connection reset" (not SSLError) in client side in such situation without unwrap in server side. The same code does not raise exception in python 2.7, which I don't know why... Any way, reading the data in server side will solve the problem, thanks. 2013/4/10 Antoine Pitrou <report@bugs.python.org> > > Antoine Pitrou added the comment: > > > The error looks like : SSLError(8, '_ssl.c:1363: EOF occurred in > > violation > > of protocol') > > But why we see "in violation of protocol" here? > > Because the SSL layer wasn't shutdown cleanly: the TCP connection was > closed while the SSL layer was still active. You have three solutions > around this: > > - you can call unwrap() for a clean SSL shutdown (the server has to call > unwrap() too). > > - you can use suppress_ragged_eofs=True with wrap_socket() > > - you can simply avoid reading past the server's data, which will > solve the problem altogether > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17672> > _______________________________________ > |
|
|
| msg186438 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2013-04-09 16:15 |
| Ok, closing the issue then. |
|
|