[Python-Dev] Adding socket timeout to urllib2 (original) (raw)
Facundo Batista facundo at taniquetil.com.ar
Mon Mar 5 16:43:43 CET 2007
- Previous message: [Python-Dev] Python-3000 upgrade path
- Next message: [Python-Dev] Adding socket timeout to urllib2
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
I studied which modifications I need to make into urllib2 to support a socket timeout.
- I'll modify urlopen for it to accept a socket_timeout parameter, default to None
- Request will also accept a socket_timeout parameter, default to None. It will keep it in a socket_timeout attribute, so it could be added/modified from outside.
- OpenerDirector's 'open' method will accept socket_timeout, and build the Request object with it
So, when it gets into the Handlers, how it'll be used? I checked the Handlers, and here is a resume of their 'open' or 'open'-like methods:
- AbstractHTTPHandler: receives a class, and uses its .request and .getresponse methods
- HTTPHandler: just calls AbstractHTTPHandler.do_open with httplib.HTTPConnection
- HTTPSHandler: just calls AbstractHTTPHandler.do_open with httplib.HTTPSConnection
- UnknownHandler: raises an Error
- FileHandler: opens a file in a local host
- FTPHandler: uses sockets in ftplib.FTP().connect, .login, .cwd, and .ntransfercmd, all through urllib's ftwrapper object
- CacheFTPHandler: instantiates ftpwrapper object; does not uses it directly, though.
- GopherHandler: is deprecated...
So, these are the necessary further modifications in urllib2 beyond those detailed at the beggining of the mail:
- Modify AbstractHTTPHandler to pass the socket_timeout parameter to the received class' methods.
- Modify FTPHandler to pass the socket_timeout to ftpwrapper class.
- Modify CacheFTPHandler the same way that FTPHandler
Beyond that, I'll also need to:
- Modify urllib.ftpwrapper to accept a socket_timeout parameter and pass it to ftplib.FTP object.
- Add support to socket_timeout to ftplib's FTP object
- Add support to socket_timeout to httplib's HTTPConnection and HTTPSConnection objects
So, I think that in the name of clarity, I shall first modify ftplib and httplib, add socket_timeout where is necessary, and then add support to that in the more general urllib2.
What do you think? Comments are very appreciated.
Thank you very much!
-- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/
- Previous message: [Python-Dev] Python-3000 upgrade path
- Next message: [Python-Dev] Adding socket timeout to urllib2
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]