Issue 4470: smtplib SMTP_SSL not working. (original) (raw)

Created on 2008-11-30 16:23 by lcatucci, last changed 2022-04-11 14:56 by admin. This issue is now closed.

| Messages (21) | | | | | | | | | | | | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------------- | --------------------------------------------------------------------------------- | -------------------------------------------- | | | ------------------------------------------------ | | --------------------- | --------------------- | -------------------------------------------------------------------------- | | msg76640 - (view) | Author: Lorenzo M. Catucci (lcatucci) * | Date: 2008-11-30 16:23 | | | | | | | | | | | The enclosed patch does three things: 1. enables SMTP_SSL working: the _get_socket method was setting self.sock instead of returning the socket to the caller, which did reset self.sock to None 2. replace home-grown SSLFakeFile() with calls to ssl.socket's makefile() calls both in the starttls and in the SMTP_SSL cases 3. shutdown sockets before closing them, to avoid server-side piling and connection refused on connection-limited servers The last change is just a cosmetical refactoring, but it really helps the SMTP_SSL case: default_port should really be a class attribute, instead of being set at __init__ time. | | | | | | | | | | | | | msg76759 - (view) | Author: Lorenzo M. Catucci (lcatucci) * | Date: 2008-12-02 16:34 | | | | | | | | | | | I've reworked the patch into a series, like haypo requested for poplib and imaplib. | | | | | | | | | | | | | msg88235 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2009-05-23 14:14 | | | | | | | | | | | With the closure of 4066 all the tests in the test patch pass, so I'm lowering the piority of this ticket. I haven't reviewed the other patches, but the tests in the test patch appear to be doing tests in the setup method, which doesn't seem like a good idea. | | | | | | | | | | | | | msg138235 - (view) | Author: Torsten Bronger (bronger) | Date: 2011-06-13 12:29 | | | | | | | | | | | I still have to apply Catucci's patch (or a modification of) after every Ubuntu installation or upgrade. Otherwise, I get ... File "/usr/lib/python2.7/smtplib.py", line 752, in __init__ SMTP.__init__(self, host, port, local_hostname, timeout) File "/usr/lib/python2.7/smtplib.py", line 239, in __init__ (code, msg) = self.connect(host, port) File "/usr/lib/python2.7/smtplib.py", line 295, in connect self.sock = self._get_socket(host, port, self.timeout) File "/usr/lib/python2.7/smtplib.py", line 757, in _get_socket new_socket = socket.create_connection((host, port), timeout) File "/usr/lib/python2.7/socket.py", line 571, in create_connection raise err socket.error: [Errno 111] Connection refused But isn't it that #4066 should have solved the critical part this issue pair 4066/4470? | | | | | | | | | | | | | msg138519 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2011-06-17 15:56 | | | | | | | | | | | Torsten, can you provide a clear, failing unittest for this? | | | | | | | | | | | | | msg138520 - (view) | Author: Torsten Bronger (bronger) | Date: 2011-06-17 16:08 | | | | | | | | | | | No, I don't know how to do that. All I can provide is a minimal version of my code that triggers the above mentioned traceback. It is: import smtplib s = smtplib.SMTP_SSL("relay-auth.rwth-aachen.de") s.login("***", "***") s.sendmail("bronger@physik.rwth-aachen.de", "bronger.randys@googlemail.com"], "Hello") I hope it helps to understand what I mean. | | | | | | | | | | | | | msg138521 - (view) | Author: Torsten Bronger (bronger) | Date: 2011-06-17 16:11 | | | | | | | | | | | Sorry, it must be: import smtplib s = smtplib.SMTP_SSL("relay-auth.rwth-aachen.de") s.login("***", "***") s.sendmail("bronger@physik.rwth-aachen.de", ["bronger.randys@googlemail.com"], "Hello") (A bracket was missing.) | | | | | | | | | | | | | msg138523 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2011-06-17 16:47 | | | | | | | | | | | According to your traceback you should be seeing the error in the first line (the creation of the SMTP_SSL object). If I run that line at the python prompt of python2.7.1, I get your connection failure. If I run it using 2.7 tip (or 3.3 tip), the connection succeeds. I know there have been other fixes in this area, but I don't know which one makes the difference. Torsten, can you test with 2.7.2 and/or 2.7 tip? I'm not sure what is left to do in this issue. Do you have an opinion, Lorenzo? | | | | | | | | | | | | | msg138552 - (view) | Author: Lorenzo M. Catucci (lcatucci) * | Date: 2011-06-17 21:09 | | | | | | | | | | | On Fri, 17 Jun 2011, R. David Murray wrote: RDM> RDM> R. David Murray <rdmurray@bitdance.com> added the comment: RDM> RDM> According to your traceback you should be seeing the error in the RDM> first line (the creation of the SMTP_SSL object). If I run that line RDM> at the python prompt of python2.7.1, I get your connection failure. RDM> If I run it using 2.7 tip (or 3.3 tip), the connection succeeds. I RDM> know there have been other fixes in this area, but I don't know which RDM> one makes the difference. RDM> RDM> Torsten, can you test with 2.7.2 and/or 2.7 tip? RDM> RDM> I'm not sure what is left to do in this issue. Do you have an RDM> opinion, Lorenzo? RDM> Torsten, would you mind letting us know both the python and the .deb version by running $ python2.7 --version and $ dpkg -l python2.7 At least in debian's python2.7_2.7.1-8, default_port is still an instance attribute instead of a class attribute. If the ubuntu situation is the same, the missing patch is smtplib_01_default_port.diff , and you could work-around the problem by explicitly calling s = smtplib.SMTP_SSL("relay-auth.rwth-aachen.de", 465). The patch got in with: changeset: 69931:bcf04ced5ef1 branch: 2.7 parent: 69915:7c3a20b5943a user: Antoine Pitrou <solipsis@pitrou.net> date: Sat May 07 19:59:33 2011 +0200 summary: Issue #11927: SMTP_SSL now uses port 465 by default as documented. Patch by Kasun Herath. The last hunk, which fixes LMTP is still missing. @@ -776,8 +777,9 @@ authentication, but your mileage might vary.""" ehlo_msg = "lhlo" + default_port = LMTP_PORT - def __init__(self, host = '', port = LMTP_PORT, local_hostname = None): + def __init__(self, host = '', port = 0, local_hostname = None): """Initialize a new instance.""" SMTP.__init__(self, host, port, local_hostname) Have a nice week-end, yours lorenzo +-------------------------+----------------------------------------------+ | Lorenzo M. Catucci | Centro di Calcolo e Documentazione | | catucci@ccd.uniroma2.it | Università degli Studi di Roma "Tor Vergata" | | | Via O. Raimondo 18 ** I-00173 ROMA ** ITALY | | Tel. +39 06 7259 2255 | Fax. +39 06 7259 2125 | +-------------------------+----------------------------------------------+ | | msg138555 - (view) | Author: Catalin Iacob (catalin.iacob) * | Date: 2011-06-17 21:32 | | | | | | | | | | | Most of the problems in this issue were solved already so it could almost be closed: * patch 1 was addressed in #11927 * patch 2 was addressed in #4066 * patches 3 and 4 were addressed in #11893 Torsten's problem was addressed by bcf04ced5ef1. > I'm not sure what is left to do in this issue. The only patch remaining is patch 5. I attached an updated version against tip of default branch. My patch mimics shutdown in imaplib.py in that it silences ENOTCONN. However I don't have a test that fails without the shutdown and I don't know if checking ENOTCONN is really needed. I tried to get shutdown to produce ENOTCONN by using Postfix as a server with smtpd_timeout=5s, connecting to it and waiting idle for more than 5 seconds before doing close(). In the Postfix logs I see that Postfix disconnects after 5 seconds of inactivity but doing shutdown afterwards doesn't trigger any exception so the ENOTCONN part remains unexercised. My patch also adds shutdown method and SHUT_RDWR constant to mock_socket.py since otherwise test_smtplib fails. (Added Antoine to nosy because he reviewed the patches for #11927 and #11893) | | | | | | | | | | | | | msg138561 - (view) | Author: Torsten Bronger (bronger) | Date: 2011-06-17 23:30 | | | | | | | | | | | My Python version is "Python 2.7.1+" and the package is called "python2.7 2.7.1-5ubuntu2" (Ubuntu Natty). | | | | | | | | | | | | | msg138581 - (view) | Author: Lorenzo M. Catucci (lcatucci) * | Date: 2011-06-18 13:02 | | | | | | | | | | | Just finished testing both 2.7 and default branches' socket close behaviour, and it seems 05 is not strictly needed. I'd still prefer if smtplib_05_shutdown_socket_v2.patch since, this way the REMOTE socket close will be unconditionally correct, instead of being dependent on GC artifacts. | | | | | | | | | | | | | msg138688 - (view) | Author: Lorenzo M. Catucci (lcatucci) * | Date: 2011-06-20 05:07 | | | | | | | | | | | I'd still prefer if smtplib_05_shutdown_socket_v2.patch could get in, ^^^^^^^^^^^^^^ since, this way the REMOTE socket close will be unconditionally correct, instead of being dependent on GC artifacts. | | | | | | | | | | | | | msg189591 - (view) | Author: Torsten Bronger (bronger) | Date: 2013-05-19 14:04 | | | | | | | | | | | For five Ubuntu releases now, I apply this patch. In contrast to Catalin's statement, it is not solved for me unless the upstream changes of two years ago haven't yet found their way into Ubuntu. I'm currently using Python 2.7.4 on Ubuntu 13.04. That said, the patch solves my issue every time. | | | | | | | | | | | | | msg189593 - (view) | Author: Antoine Pitrou (pitrou) * | Date: 2013-05-19 14:11 | | | | | | | | | | | When you say "I apply this patch", you mean smtplib_05_shutdown_socket_v2.patch, right? | | | | | | | | | | | | | msg189594 - (view) | Author: Antoine Pitrou (pitrou) * | Date: 2013-05-19 14:12 | | | | | | | | | | | In any case, I'm growing wary of bugfix regressions right now, so I would only apply the patch on the default branch. | | | | | | | | | | | | | msg189597 - (view) | Author: Torsten Bronger (bronger) | Date: 2013-05-19 14:49 | | | | | | | | | | | Sorry, after having had another look at it, I realised that I have a different SSMTP issue now, non-Python-related. So for me, Ubuntu 13.04 indeed solves my old issue. | | | | | | | | | | | | | msg189617 - (view) | Author: R. David Murray (r.david.murray) * | Date: 2013-05-19 18:52 | | | | | | | | | | | Lorenzo, any chance you could supply a unit test that fails without smtplib_05_shutdown_socket.diff applied? | | | | | | | | | | | | | msg189619 - (view) | Author: Lorenzo M. Catucci (lcatucci) * | Date: 2013-05-19 19:07 | | | | | | | | | | | On Sun, 19 May 2013, R. David Murray wrote: RDM> RDM> R. David Murray added the comment: RDM> RDM> Lorenzo, any chance you could supply a unit test that fails without RDM> smtplib_05_shutdown_socket.diff applied? RDM> It would really be a functional test, since the problem with half-open connection pile-up stems from smtp server's access control rules. To test we should setup a fake smtp server, which forbids having multiple connections from the same IP address, and connect twice in a row to the fake server. I'm not sure I'm able to implement both an smtpd.py server serving more than one connection and the client code in a race-free way in the same "unit" test. Will try in the next week. Thank you very much, lorenzo | | | | | | | | | | | | | msg275022 - (view) | Author: Christian Heimes (christian.heimes) * | Date: 2016-09-08 14:38 | | | | | | | | | | | The bug is 8 years old and hasn't seen activity for three years. Is SMTP over SSL still broken for you? | | | | | | | | | | | | | msg316120 - (view) | Author: Cheryl Sabella (cheryl.sabella) * | Date: 2018-05-03 11:49 | | | | | | | | | | | This issue has been in pending status for over 18 months following Christian's question to the OP, so closing as out of date. | | | | | | | | | | | |