Issue 2868: Problem with urllib and urllib2 in urlopen? (original) (raw)

Issue2868

Created on 2008-05-15 17:34 by ambarish, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (4)
msg66872 - (view)	Author: Ambarish Malpani (ambarish)	Date: 2008-05-15 17:34
I have the following code: import urllib u = 'http://www.mercurynews.com/ci_9216417' h = urllib.urlopen(u).read() print h # Get an empty string #(can use urllib2 also - get the same behavior) If I visit the same page with my browser, get the contents of the page (after some redirects...)
msg66881 - (view)	Author: Benjamin Peterson (benjamin.peterson) *	Date: 2008-05-15 21:45
This is what happens on the trunk: >>> import urllib >>> u = 'http://www.mercurynews.com/ci_9216417' >>> h = urllib.urlopen(u).read() Traceback (most recent call last): File "", line 1, in File "/temp/python/trunk/Lib/ssl.py", line 333, in read data = self._sslobj.read(recv_size) ssl.SSLError: [Errno 8] _ssl.c:1276: EOF occurred in violation of protocol
msg66888 - (view)	Author: Senthil Kumaran (orsenthil) *	Date: 2008-05-16 02:36
Here is my analysis: >>> import urllib2 >>> url = "http://www.mercurynews.com/ci_9216417" >>> content = urllib2.urlopen(url).read() >>> print content >>> opened = urllib2.urlopen(url) >>> print opened.geturl() https://secure.passport.mnginteractive.com/mngi/servletDispatch/ErightsPassportServlet.dyn?url=http://www.mercurynews.com/ci_9216417?nclick_check=1&forced=true # This URL Redirection is a 302 Redirection. # Browser is "unable" to launch the redirected site. https://secure.passport.mnginteractive.com/mngi/servletDispatch/ErightsPassportServlet.dyn?url=http://www.mercurynews.com/ci_9216417?nclick_check=1&forced=true # Logically, the urllib /urllib2 is giving a Blank when reading this site. I would't entire say urllib2's fault before understanding how FF is handling the redirection of the first site. 1) Open the site mentioned in the Location of 302, you will experience the same behaviour as 302. It seems more of an issue at server end, we have to know how, Firefox is handling at the first place.
msg67003 - (view)	Author: Martin McNickle (BitTorment)	Date: 2008-05-17 15:59
I verified the behaviour but this is a problem with that particular site, not with urllib/urllib2. Should be closed.