| msg159915 - (view) |
Author: Arve Knudsen (Arve.Knudsen) |
Date: 2012-05-04 10:06 |
| httplib doesn't specify the HTTP header 'content-length' for POST requests without data. Conceptually this makes sense, considering the empty content. However, IIS (7.5) servers don't accept such requests and respond with a 411 status code. See this question on StackOverflow for reference: http://stackoverflow.com/questions/5915131/can-i-send-an-empty-http-post-webrequest-object-from-c-sharp-to-iis. See also issue #223 of the Requests project, https://github.com/kennethreitz/requests/issues/223, which regards this problem in the context of the requests Python library. The following code makes a data-less POST request to the HTTP sniffer Fiddler running on localhost: import httplib conn = httplib.HTTPConnection("localhost", 8888) conn.request("POST", "/post", "", {}) conn.close() Fiddler reports that it receives the following headers for the POST request, as you can see 'content-length' is not included: POST http://localhost:8888/post HTTP/1.1 Host: localhost:8888 Accept-Encoding: identity |
|
|
| msg159935 - (view) |
Author: Jesús Cea Avión (jcea) *  |
Date: 2012-05-04 14:21 |
| Could you provide a patch? Does this affect 3.x too? |
|
|
| msg159936 - (view) |
Author: Arve Knudsen (Arve.Knudsen) |
Date: 2012-05-04 14:35 |
| I can look into patch and 3.x tonight I think. Should I provide a test with an eventual patch? |
|
|
| msg159938 - (view) |
Author: Jesús Cea Avión (jcea) * |
Date: 2012-05-04 14:42 |
| Patch with test, please :-). I know it is a pain in the ass, but the result is having a higher quality python. |
|
|
| msg159942 - (view) |
Author: Piotr Dobrogost (piotr.dobrogost) |
Date: 2012-05-04 16:01 |
| > Fiddler reports that it receives the following headers for the POST request Python 3.2.3 on Windows Vista 64bit gives the same output for import http.client conn = http.client.HTTPConnection('localhost',8888) conn.request("POST", "/post", "", {}) conn.close() |
|
|
| msg159946 - (view) |
Author: Arve Knudsen (Arve.Knudsen) |
Date: 2012-05-04 17:23 |
| Which HTTP methods should we auto-define content-length for? POST and PUT? I noticed that my first attempt at a patch would define content-length also for GET requests, which broke a unit test (test_ipv6host_header). |
|
|
| msg159954 - (view) |
Author: Arve Knudsen (Arve.Knudsen) |
Date: 2012-05-04 18:12 |
| Actually, when inspecting the HTTP requests sent by Chrome for the different methods (a great little Chrome app called Postman let me fire requests manually), I found that content-length would be set for most methods. I could confirm that 'content-length: 0' would be set for the following methods: POST, PUT, PATCH, DELETE and HEAD. I guess it should be good enough to model that behaviour in httplib? |
|
|
| msg159959 - (view) |
Author: Jesús Cea Avión (jcea) * |
Date: 2012-05-04 19:12 |
| HEAD?. It doesn't make sense in HEAD if it doesn't make sense in GET. Looking around, I found this, to mud the water a little bit more: <http://fixunix.com/tcp-ip/66198-http-rfc-related-question-content-length-0-get-request.html>. Not being explicit about this is a "bug" in the specification, I think. |
|
|
| msg159964 - (view) |
Author: Arve Knudsen (Arve.Knudsen) |
Date: 2012-05-04 19:34 |
| Here's my initial proposal for a patch against httplib, based on the 2.7 branch of cpython repository. I've included a couple of tests, which check that when content is empty, content-length is set to 0 for certain methods (POST/PUT etc) and not specified for others. |
|
|
| msg159965 - (view) |
Author: Arve Knudsen (Arve.Knudsen) |
Date: 2012-05-04 19:34 |
| Yes, I agree it doesn't make much sense for HEAD AFAICT, but Chrome does it. Maybe there's a reason? |
|
|
| msg161096 - (view) |
Author: Senthil Kumaran (orsenthil) * |
Date: 2012-05-19 08:05 |
| The rule for content-length seems, if there is a body for a request, even if the body is "" ( empty body), then you should send the Content-Length. The mistake in the Python httplib was, the set_content_length was called with this condition. if body and ('content-length' not in header_names): If the body was '', this was skipped. The default for GET and methods which do not use body was body=None and that was statement for correct in those cases. A simple fix which covers the applicable methods and follows the definition of content-length seems to me like this: - if body and ('content-length' not in header_names): + if body is not None and 'content-length' not in header_names: I prefer this rather than checking for methods explicitly as it could go into unnecessary details. (Things like if you are not sending a body why are you sending a Content-Length?. This fails the definition of Content-Length itself). The Patch is fine, I would adopt that for the above check and commit it all the active versions. Thanks Arve Knudsen, for the bug report and the patch. |
|
|
| msg161098 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2012-05-19 08:59 |
| New changeset 57f1d13c2cd4 by Senthil Kumaran in branch '2.7': Fix Issue14721: Send Content-length: 0 for empty body () in the http.request http://hg.python.org/cpython/rev/57f1d13c2cd4 New changeset 6da1ab5f777d by Senthil Kumaran in branch '3.2': Fix Issue14721: Send Content-length: 0 for empty body () in the http.client requests http://hg.python.org/cpython/rev/6da1ab5f777d New changeset 732d70746fc0 by Senthil Kumaran in branch 'default': merge - Fix Issue14721: Send Content-length: 0 for empty body () in the http.client requests http://hg.python.org/cpython/rev/732d70746fc0 |
|
|
| msg161099 - (view) |
Author: Senthil Kumaran (orsenthil) * |
Date: 2012-05-19 09:00 |
| This is fixed in all the branches. Thanks! |
|
|
| msg161154 - (view) |
Author: Jesús Cea Avión (jcea) * |
Date: 2012-05-19 19:46 |
| Too late for asking to keep the parenthesis :-). I hate to have to remember non-obvious precedence rules :-). Cognitive overhead. |
|
|
| msg236600 - (view) |
Author: James Rutherford (jimr) * |
Date: 2015-02-25 16:59 |
| The fix for this still doesn't set Content-Length to zero when body is None, but I don't see any reason why this should be the case. For example, the following snippet would work for any 'empty' body: if 'content-length' not in header_names: self._set_content_length(body if body is not None else '') I'm happy to produce a patch if there's any chance it would be merged. |
|
|
| msg236686 - (view) |
Author: Demian Brecht (demian.brecht) * |
Date: 2015-02-26 16:41 |
| > I'm happy to produce a patch if there's any chance it would be merged. If the patch adheres to the RFC, then I see no reason why it shouldn't be merged. What makes this a little more tricky than the snippet that you included in your post though (which would include the Content-Length header for all HTTP methods) is the following from RFC 7230: A user agent SHOULD send a Content-Length in a request message when no Transfer-Encoding is sent and the request method defines a meaning for an enclosed payload body. For example, a Content-Length header field is normally sent in a POST request even when the value is 0 (indicating an empty payload body). A user agent SHOULD NOT send a Content-Length header field when the request message does not contain a payload body and the method semantics do not anticipate such a body. Currently, there is nothing in the http package that defines whether or not a given HTTP method expects a body (as far as I'm aware at any rate), although this would be a simple addition. I'd imagine that the result might look like this: _METHODS_EXPECTING_BODIES = {'OPTIONS', 'POST', 'PUT', 'PATCH'} if method.upper() in _METHODS_EXPECTING_BODIES and \ 'content-length' not in header_names: self._set_content_length(body) I'd prefer to have the conversion from None to empty string done in the body of _set_content_length in order to ensure consistency should the call be made from elsewhere. |
|
|
| msg236731 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2015-02-27 08:19 |
| James, Demian: this issue has been closed for almost three years and the changes released long ago: comments made here will likely be ignored. Please open a new issue if you want them to be acted on. |
|
|
| msg236804 - (view) |
Author: Demian Brecht (demian.brecht) * |
Date: 2015-02-27 17:31 |
| Thanks for the heads up Ned. James: I've created #23539 in the event that you'd like to contribute a patch. |
|
|