[Python-Dev] Suggested changes to verify HTTPS by default (was Re: Proposed schedule for 3.4.2) (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Tue Sep 9 12:52:12 CEST 2014

On 9 September 2014 03:44, Alex Gaynor <alex.gaynor at gmail.com> wrote:

Shifts uncomfortably it looks like presently there's not a good way to change anything about the SSL configuration for urllib.request.urlopen. It does not take a context argument, as the http.client API does: https://docs.python.org/3/library/urllib.request.html#module-urllib.request and instead takes the cafile, capath, cadefault args.

This would need to be updated first, once it did take such an argument, this would be accomplished by: context = ssl.createdefaultcontext() context.verifymode = CERTOPTIONACERTNONE context.verifyhostname = False urllib.request.urlopen("https://something-i-apparently-dont-care-much-about", context=context)

I'd never needed to use the existing global configuration settings in urllib.request before, but it actually does already support setting the default opener for urllib.urlopen.

To explicitly set it to use verified HTTPS by default:

import ssl, urllib.request
https_handler = HTTPSHandler(context=ssl.create_default_context(),

check_hostname=True) urllib.request.install_opener(urllib.request.build_opener(https_handler)

When the default changes, turning off verification by default for urllib.request.urlopen would look like:

import ssl, urllib.request
unverified_context = ssl.create_default_context()
unverified_context.verify_mode = CERT_OPTIONACERT_NONE
unverified_context.verify_hostname = False
unverified_handler = HTTPSHandler(context=unverified_context,

check_hostname=False) urllib.request.install_opener(urllib.request.build_opener(unverified_handler)

However, even setting the opener like that still leaves http.client.HTTPSConnection, urllib.request.URLOpener and urllib.request.FancyURLOpener using unverified HTTPS with no easy way to change their default behaviour.

That means some other approach to global configuration is going to be needed to cover the "coping with legacy corporate infrastructure" case, and I still think a monkeypatching based hack is likely to be our best bet.

So, focusing on 3.4, but in a way that should be feasible to backport, the changes that I now believe would be needed are:

  1. Add "context" arguments to urlopen, URLOpener and FancyURLOpener (the latter two have been deprecated since 3.3, but this would make things easier for a subsequent 2.7 backport)
  2. Add a ssl._create_https_context() alias for ssl.create_default_context()
  3. Change urllib.request.urlopen() and http.client.HTTPSConnection to call ssl_create_https_context() rather than ssl._create_stdlib_context()
  4. Rename ssl._create_stdlib_context() to ssl._create_unverified_context() (updating call sites accordingly)

To revert any given call site to the old behaviour:

http.client.HTTPSConnection(context=ssl._create_unverified_context())
urllib.request.urlopen(context=ssl._create_unverified_context())
urllib.request.URLOpener(context=ssl._create_unverified_context())
urllib.request.FancyURLOpener(context=ssl._create_unverified_context())

And to revert to the old default behaviour globally:

import ssl
ssl._create_https_context = ssl._create_unverified_context

The backport to 2.7 would then be a matter of bringing urllib, urllib2, httplib and ssl into line with their 3.4.2 counterparts.

Regards, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list