Issue 9003: urllib.request and http.client should allow certificate checking (original) (raw)

process

Status: closed Resolution: fixed
Dependencies: 1589 Superseder:
Assigned To: orsenthil Nosy List: Ryan.Tucker, ahasenack, asdfasdfasdfasdfasdfasdfasdf, debatem1, devin, giampaolo.rodola, heikki, janssen, jsamuel, kiilerix, orsenthil, pitrou, python-dev, vila, zooko
Priority: normal Keywords: patch

Created on 2010-06-16 01:04 by debatem1, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
httpcli.patch pitrou,2010-10-08 16:56
httpcli2.patch pitrou,2010-10-09 15:09
httpcli+urllib.patch pitrou,2010-10-11 16:34
httpcli+urllib2.patch pitrou,2010-10-11 17:47
httpcli+urllib3.patch pitrou,2010-10-11 19:48
Messages (13)
msg107900 - (view) Author: geremy condra (debatem1) Date: 2010-06-16 01:04
urllib currently blindly accepts bad certificates when passed an https address. This behavior, clearly not desirable for many users, is also not documented. I propose one of two changes: 1) add mechanisms for enforcing correct behavior to urllib, or 2) change the documentation for that module to include something akin to the following warning: "Warning: urllib does not perform certificate checks if passed an HTTPS url! This permits remote machines to masquerade as your intended destination."
msg118081 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-06 20:52
A big warning is now present (*) in the urllib and httplib documentation pages. Also, once is fixed, we can go forward and make {http.client,urllib.request} check hostname and cert if the user gives the location of a bunch of CA certs. (*) see e.g. http://docs.python.org/dev/library/urllib.request.html
msg118178 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-08 11:03
Here is the API addition I would suggest for the http.client module: Add two new keyword arguments `context` and `check_hostname` to HTTPSConnection; `context` would allow to pass a SSLContext instance for certificate checking and other options (default None, meaning no checking); `check_hostname` would specify whether to check the hostname against the URL (default to check only if context is present and context.verify_mode != CERT_NONE). Here is the API addition I would suggest for the urllib.request module: - Add constructor arguments `context` and `check_hostname` to HTTPSHandler. They will be passed to the underlying HTTPSConnection. - Add `ssl_ca_file` and `ssl_ca_path` arguments to the high-level function urlopen(); if at least one of them is present, a custom opener with a custom HTTPSHandler will be created, mandating the checking of server certificates
msg118206 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-08 16:56
Here is a preliminary patch for http.client. I think it would be good to have local tests using a custom HTTPS server, too.
msg118276 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-09 15:09
Here is another patch for http.client containing more tests, including with a mismatching cert. Comments welcome.
msg118291 - (view) Author: geremy condra (debatem1) Date: 2010-10-09 21:34
Any chance on folding the HTTPSServer class into http.server? Geremy Condra
msg118292 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-09 21:56
> Any chance on folding the HTTPSServer class into http.server? Its API and implementation would first have to be cleaned up. I'd prefer if it were the subject of a separate issue.
msg118379 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-11 16:34
Here is a patch which also adds 'cafile' and 'capath' keyword arguments to urlopen().
msg118381 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-11 17:47
Here is a new patch with doc updates for urllib.request.
msg118383 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-11 19:48
This patch should fix the test hanging issues witnessed on some machines.
msg118391 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2010-10-11 20:35
Yes, it does solve the problem of httplib and urllib2_localnet tests which were hanging with the earlier patch on certain machines..
msg118510 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-13 10:40
Patch committed in r85408. I believe this fixes, at last, the whole issue people were complaining about.
msg231573 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-11-23 17:46
New changeset 1882157b298a by Benjamin Peterson in branch '2.7': allow passing cert/ssl information to urllib2.urlopen and httplib.HTTPSConnection https://hg.python.org/cpython/rev/1882157b298a
History
Date User Action Args
2022-04-11 14:57:02 admin set github: 53249
2014-11-23 17:46:30 python-dev set nosy: + python-devmessages: +
2010-11-11 15:55:14 pitrou link issue8804 superseder
2010-10-13 10:40:17 pitrou set status: open -> closedversions: - Python 3.3messages: + resolution: fixedstage: patch review -> resolved
2010-10-12 18:06:41 pitrou set files: - unnamed
2010-10-11 20:35:39 orsenthil set files: + unnamedmessages: +
2010-10-11 19:48:53 pitrou set files: + httpcli+urllib3.patchmessages: +
2010-10-11 17:47:19 pitrou set files: + httpcli+urllib2.patchmessages: +
2010-10-11 16:34:21 pitrou set files: + httpcli+urllib.patchmessages: + stage: needs patch -> patch review
2010-10-09 21:56:55 pitrou set messages: +
2010-10-09 21:34:16 debatem1 set messages: +
2010-10-09 15:09:40 pitrou set files: + httpcli2.patchmessages: +
2010-10-08 16:56:48 pitrou set files: + httpcli.patchkeywords: + patchmessages: +
2010-10-08 11:03:36 pitrou set resolution: accepted -> (no value)messages: +
2010-10-06 20:52:27 pitrou set type: enhancementdependencies: + New SSL module doesn't seem to verify hostname against commonName in certificatetitle: urllib about https behavior -> urllib.request and http.client should allow certificate checkingnosy: + zooko, janssen, vila, heikki, ahasenack, kiilerix, devin, asdfasdfasdfasdfasdfasdfasdf, Ryan.Tuckerversions: + Python 3.2, Python 3.3, - Python 3.1messages: + stage: needs patch
2010-06-16 16:54:21 jsamuel set nosy: + jsamuel
2010-06-16 14:24:26 giampaolo.rodola set nosy: + giampaolo.rodola
2010-06-16 03:08:31 r.david.murray set nosy: + pitrou
2010-06-16 01:20:55 orsenthil set assignee: orsenthilresolution: acceptednosy: + orsenthil
2010-06-16 01:04:36 debatem1 create