[Python-Dev] [ 1673007 ] urllib2 requests history + HEAD (original) (raw)

KoDer koder.mail at gmail.com
Tue Mar 13 11:40:53 CET 2007

From: Facundo Batista <facundo at taniquetil.com.ar> This patch was posted by "koderua".

I think that Request must have a "request type" parameters, so people can send "HEAD" requests easily.

But it seems to me that keeping a request history in the module is bad, because it can easily grow up to thousands and explode (a.k.a. consume too much memory).

Fo example, I have a web service, running 7x24, and opening another web service, with around 10 requests per second. This means, keeping the history (around 50bytes each request), 1.2 GB of RAM in only a month!

So, I'll close this patch as "Rejected", for this reason, if anyone raises objections.

Regards, -- . Facundo

This is probably a misunderstanding. Request's history don't store in the "module".They store in two places:

  1. In Request object (for current request, so they would be destroyеd with it);
  2. In HTTPConnection object (while request redirects). In HTTPConnection history stores only for current served Request. Even if You use the same HTTPConnection for many Requests, they (HTTPConnection) clear history every time when new Request starts.

from httplib HTTPConnection.putrequest patched

str = '%s %s %s' % (method, url, self._http_vsn_str) self._output(str) self.sended_hdrs = [str] <<< previous history die here

Full history for all processed request didn't not stored in any place_.

KDanilov aka koder(aka koder_ua)

More information about the Python-Dev mailing list