Issue 12226: use HTTPS by default for uploading packages to pypi (original) (raw)
Created on 2011-05-31 16:51 by techtonik, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Messages (48)
Author: anatoly techtonik (techtonik)
Date: 2011-05-31 16:51
Before the next version is released, I'd like to push this one line modification to reduce the risk of sniffing Python development password when people upload packages to PyPI by using https:// communication channel by default.
Distutils doesn't validate PyPI server certificate, so this change doesn't prevent from MITM attacks, but at least it makes package submissions over wireless channels and public networks safer.
Taking into account that people still release packages for Python 2.5+ (AppEngine), I'd like to see this fix backported to at least Python 2.6
Author: Brian Curtin (brian.curtin) * 
Date: 2011-05-31 16:51
This should probably be discussed on catalog-SIG, not the CPython bug tracker.
Author: Brian Curtin (brian.curtin) *
Date: 2011-05-31 16:52
Oops, nevermind that, thought this was suggesting a change to PyPI itself, not distutils.
Author: anatoly techtonik (techtonik)
Date: 2011-05-31 16:54
The distutils version that exposes this vulnerability is shipped with Python, that's why it is here.
Author: Tarek Ziadé (tarek) *
Date: 2011-05-31 19:13
Have you tried registering/uploading stuff through https ? if it's working as is I am +1 for this change in the next 2.7.x.
For 2.6 don't know if it's worth a security patch. Martin, Barry, any opinion ?
Author: anatoly techtonik (techtonik)
Date: 2011-05-31 20:15
Yes. I've just uploaded http://pypi.python.org/pypi/review/r749 through https://pypi.python.org/pypi
Author: Martin v. Löwis (loewis) *
Date: 2011-06-01 05:23
I'm +0 on changing 2.6. Many people use setuptools or distribute, so the impact of this change was higher if those were modified instead of modifying 2.6 - in particular since few people are expected to pick up these security releases, anyway.
Author: Stefan Krah (skrah) *
Date: 2011-06-01 07:30
Distutils doesn't validate PyPI server certificate, so this change doesn't prevent from MITM attacks, but at least it makes package submissions over wireless channels and public networks safer.
Is that so? It's been a while, but I think e.g. ettercap is a highly automated tool for MITM attacks that isn't very hard to use.
Author: Barry A. Warsaw (barry) *
Date: 2011-06-01 11:06
Given that 2.6.7 is rc2 with a final release scheduled in 2 days, I don't want to apply this to 2.6 right now. Can you guarantee this won't regress for anybody? If so, then I'm also +0 for 2.6 after the 2.6.7 release.
Author: anatoly techtonik (techtonik)
Date: 2011-06-01 15:11
On Wed, Jun 1, 2011 at 10:30 AM, Stefan Krah <report@bugs.python.org> wrote:
Distutils doesn't validate PyPI server certificate, so this change doesn't prevent from MITM attacks, but at least it makes package submissions over wireless channels and public networks safer.
Is that so? It's been a while, but I think e.g. ettercap is a highly automated tool for MITM attacks that isn't very hard to use.
This patch won't help against properly baited ettercap, but will prevent transit sniffing of weakly protected passwords.
anatoly t.
Author: anatoly techtonik (techtonik)
Date: 2011-06-01 15:14
On Wed, Jun 1, 2011 at 2:06 PM, Barry A. Warsaw <report@bugs.python.org> wrote:
Barry A. Warsaw <barry@python.org> added the comment:
Given that 2.6.7 is rc2 with a final release scheduled in 2 days, I don't want to apply this to 2.6 right now. Can you guarantee this won't regress for anybody? If so, then I'm also +0 for 2.6 after the 2.6.7 release.
Adding catalog-sig to CC. I can guarantee this for Windows. I'll be near Linux box tomorrow and will try upload to PyPI from there. It still will be more authoritative if more than one person can test upload to PyPI with this patch on different systems.
anatoly t.
Author: Fred Drake (fdrake)
Date: 2011-06-01 15:17
On Wed, Jun 1, 2011 at 11:14 AM, anatoly techtonik <techtonik@gmail.com> wrote:
Adding catalog-sig to CC. I can guarantee this for Windows. I'll be near Linux box tomorrow and will try upload to PyPI from there. It still will be more authoritative if more than one person can test upload to PyPI with this patch on different systems.
The interesting case will be for a build that doesn't include SSL support.
-Fred
Author: Tarek Ziadé (tarek) *
Date: 2011-06-01 15:22
On Wed, Jun 1, 2011 at 5:17 PM, Fred Drake <fdrake@acm.org> wrote:
On Wed, Jun 1, 2011 at 11:14 AM, anatoly techtonik <techtonik@gmail.com> wrote:
Adding catalog-sig to CC. I can guarantee this for Windows. I'll be near Linux box tomorrow and will try upload to PyPI from there. It still will be more authoritative if more than one person can test upload to PyPI with this patch on different systems.
The interesting case will be for a build that doesn't include SSL support.
Yeah.. We do have in packaging a way to test against a PyPI server that gets launched on a real socket, so what we could do is:
1/ add a test that tries to upload and register via ssh 2/ make sure it fallbacks to http if _ssl is not found
-Fred
-- Fred L. Drake, Jr. "Give me the luxuries of life and I will willingly do without the necessities." --Frank Lloyd Wright
Catalog-SIG mailing list Catalog-SIG@python.org http://mail.python.org/mailman/listinfo/catalog-sig
Author: Éric Araujo (eric.araujo) *
Date: 2011-06-03 16:22
Patch needs to update the default server in packaging too.
Author: Stefan Krah (skrah) *
Date: 2011-06-03 18:39
I think there should be a warning that the connection is unauthenticated (i.e. not secure). Users tend to be upset if they see 'https' and later find out that no certificates were verified.
A reasonably secure alternative is to publish the pypi server certificate in a couple of places (python-dev, www.python.org). Then the user can import the certificate into the browser while on a trusted connection and henceforth do all uploading etc. via the browser.
Author: Martin v. Löwis (loewis) *
Date: 2011-06-03 19:47
If users use a browser to do secure uploading, there is no need to publish the certificate. It is signed by a trusted CA (cacert), so you just need to make sure your browser knows about the cacert certificate.
Author: anatoly techtonik (techtonik)
Date: 2011-06-03 20:52
Sorry for the delay. I've just uploaded http://pypi.python.org/pypi/ctypesgen/0.r125 from Ubuntu using python2.6 with patched distutils module to https://pypi.python.org/pypi and can confirm it works without problems on Linux.
So I can guarantee there won't be any regress in 99.9% of cases and <0.1% of cases that are left, risk of breaking these flaky setups doesn't outweight the need to close this security hole.
I can fill separate bug for packaging, server certificate validation and support for non-SSL builds of python (for which there is workaround with upload -r option that I've just discovered), but it is an additional delay and the only reason I invested some time into this issue is because I saw RC announcements.
P.S. To Martin. CAcert is not trusted authority on Windows, and all browsers warn about it, so we need to distribute its root certificate with Python if we want to validate ours.
Author: Martin v. Löwis (loewis) *
Date: 2011-06-03 22:59
I can fill separate bug for packaging, server certificate validation and support for non-SSL builds of python (for which there is workaround with upload -r option that I've just discovered), but it is an additional delay and the only reason I invested some time into this issue is because I saw RC announcements.
This effort was wasted - there is zero chance that this can go into a release candidate. Only fundamental flaws (such as being unable to compile Python on a major platform, or serious regressions) should be considered for changes between a rc and the final release.
Author: anatoly techtonik (techtonik)
Date: 2011-06-04 05:01
I believe that's a very personal judgement. For me exposing core Python development accounts is a fundamental flaw. The more accounts are collected, the more real are attacks through PyPI package injection.
Author: Éric Araujo (eric.araujo) *
Date: 2011-06-04 14:33
I think there should be a warning that the connection is unauthenticated (i.e. not secure). Users tend to be upset if they see 'https' and later find out that no certificates were verified.
Thanks Stephan, that was on my mind but I forgot it. I’m -1 on using https if no validation is performed.
I believe that's a very personal judgement.
Not really; it’s an explanation of our release rules, exposed by one of the older developers.
For me exposing core Python development accounts is a fundamental flaw.
What is a core Python development account?
Author: anatoly techtonik (techtonik)
Date: 2011-06-06 07:54
On Sat, Jun 4, 2011 at 5:33 PM, Éric Araujo <report@bugs.python.org> wrote:>
I think there should be a warning that the connection is unauthenticated (i.e. not secure). Users tend to be upset if they see 'https' and later find out that no certificates were verified.
Thanks Stephan, that was on my mind but I forgot it. I’m -1 on using https if no validation is performed.
It will be more professional if you could also explain why. Thanks.
I believe that's a very personal judgement. Not really; it’s an explanation of our release rules, exposed by one of the older developers.
Release rules should be clear enough not to require explanation.
For me exposing core Python development accounts is a fundamental flaw.
What is a core Python development account?
'core' is not the best word here, so it needs an explanation. Any account on PyPI that uploads packages used for in enterprise deployment schemes imposes a danger. Potential target are identified using 'popularity package/developer activity' rating to reduce the risk. These are the primary targets for an attack, which I called 'core'. 'primary' would be a better name probably.
Author: Éric Araujo (eric.araujo) *
Date: 2011-06-06 15:50
Thanks Stephan, that was on my mind but I forgot it. I’m -1 on using https if no validation is performed. It will be more professional if you could also explain why.
If you make an HTTPS connection without checking the certificate, what security does it add?
Not really; it’s an explanation of our release rules, exposed by one of the older developers. Release rules should be clear enough not to require explanation.
Explanations make them clear.
Any account on PyPI that uploads packages used for in enterprise deployment schemes imposes a danger.
Sidenote: I don’t want to give less security to non-enterprise users.
Anyway, I understand your point now: insecure upload and download are vulnerable to MITM attacks, and encouraging HTTPS use (through default value + docs) would help against that. I am supportive of a patch, but it doesn’t mean the release process should not be followed. See also #11357 and #8561 about download security.
Author: Fred Drake (fdrake)
Date: 2011-06-06 16:01
On Mon, Jun 6, 2011 at 11:50 AM, Éric Araujo <report@bugs.python.org> wrote:
If you make an HTTPS connection without checking the certificate, what security does it add?
I'm in favor of cert checking, myself.
Author: Antoine Pitrou (pitrou) *
Date: 2011-06-07 11:16
If you make an HTTPS connection without checking the certificate, what security does it add?
Well, it does prevent the most trivial class of attacks (sniffing). That said, Python has support for certificate checking, especially in 3.2+, so you should use that. You could e.g. bundle the CACert root certificate with the distribution.
Author: anatoly techtonik (techtonik)
Date: 2011-06-18 09:26
This simple patch slipped off 2.7.2. Why?
Author: Éric Araujo (eric.araujo) *
Date: 2011-06-18 20:52
Because it’s not finished.
Author: anatoly techtonik (techtonik)
Date: 2011-06-18 21:27
What is left?
Author: Éric Araujo (eric.araujo) *
Date: 2011-06-18 21:29
Certificate checking.
Author: anatoly techtonik (techtonik)
Date: 2011-06-18 21:55
That's the .
Author: Christian Heimes (christian.heimes) *
Date: 2013-02-16 00:10
CVE-2013-1754 Man-in-the-middle vulnerability in package upload feature of Python's distutils
Author: Giovanni Bajo (Giovanni.Bajo)
Date: 2013-02-24 11:52
Please notice that a redesign of PyPI and package security is ongoing in catalog-sig.
Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * 
Date: 2013-03-18 23:44
New changeset f86d46a580d8 by Benjamin Peterson in branch 'default': use the HTTPS for pypi upload http://hg.python.org/cpython/rev/f86d46a580d8
Author: Éric Araujo (eric.araujo) *
Date: 2013-03-24 05:59
Benjamin, you committed a change to use HTTPS instead of HTTP. In this bug report, we were having a discussion about the false/incomplete security that this provides if there is no certificate checking. What are your thoughts on that?
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2013-03-24 12:31
This is true, but if we get proper certificate checking, this should automatically work correctly then.
Author: Éric Araujo (eric.araujo) *
Date: 2013-03-25 03:11
I’m not sure what “this” refers to (in “This is true” and “this should automatically work correctly”).
My only concern is to avoid giving a false sense of security, so my initial stance was all-or-nothing. However with the recent trend of incremental improvements to the PyPI ecosystem, I think it’s important to do what we can and keep the momentum, so I’m okay with the commit—I just wanted to make sure that committing half a fix was intentional. You probably know more about SSL than me and you’re the RM, so let’s ship this. :)
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2013-03-25 03:23
By "this", I meant the change I made. It was made in consultation with Richard Jones (added to nosy) at the PyCon sprints.
Author: Donald Stufft (dstufft) *
Date: 2013-03-25 03:57
Using HTTPS without a Certificate prevents passive attacks but not active attacks. It puts things in a better situation but not the ideal situation.
Author: anatoly techtonik (techtonik)
Date: 2013-06-08 21:46
This should have been backported to Python 2. I expect some related attacks on EuroPython.
Author: Donald Stufft (dstufft) *
Date: 2013-06-08 22:18
I would +! backporting this, but It's not massively required since it only protects against passive attacks.
It would however make things a little nicer.
Author: anatoly techtonik (techtonik)
Date: 2013-06-08 22:26
If somebody sponsor my visit to EuroPython, I will dedicate some time to prepare a demo uploading rogue packages using sniffed credentials over WiFi without owner's consent. After moving to CDN no upload logs are available, so it is even more secure for attacker to do this stuff.
Author: Donald Stufft (dstufft) *
Date: 2013-06-08 22:28
Uploading always hits the backend servers and thus has the same logging as before
Merely switching to HTTPS only provides protections against passive attacks. You need verification to protect against active attacks (which are simple and easy to do as well). Like I said, not a bad move and I'd be in agreement on doing it but if the powers that be decide it's too big of a change it's not going to massively decrease the security.
Author: anatoly techtonik (techtonik)
Date: 2013-11-10 19:01
How come that this CVE is still present in just released 2.7.6?
Author: Éric Araujo (eric.araujo) *
Date: 2013-11-22 21:08
Donald assesses that porting the changeset to 2.7 would “make things a little nicer”, as it protects from passive attacks only. The change is small. What do people think?
Author: Antoine Pitrou (pitrou) *
Date: 2013-11-22 21:10
Well, passive attacks are the easiest to mount by a casual attacker, so I think this is important to get in.
Author: Christian Heimes (christian.heimes) *
Date: 2013-11-22 21:25
How about:
- load ca cert from default verify locations
- try connect with CERT_REQUIRED
- print warning when cert validation fails and try again with CERT_NONE
- match hostname otherwise
At least this warns the user about the issue. Is there way to distinguish between CA missing and other failures?
Antoine Pitrou <report@bugs.python.org> schrieb:
Antoine Pitrou added the comment:
Well, passive attacks are the easiest to mount by a casual attacker, so I think this is important to get in.
Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12226>
Author: Roundup Robot (python-dev)
Date: 2013-12-22 00:36
New changeset 32a39ec6bd75 by Antoine Pitrou in branch '2.7': Issue #12226: HTTPS is now used by default when connecting to PyPI. http://hg.python.org/cpython/rev/32a39ec6bd75
Author: Roundup Robot (python-dev)
Date: 2013-12-22 00:48
New changeset 2b5cd6d4d149 by Antoine Pitrou in branch '3.2': Issue #12226: HTTPS is now used by default when connecting to PyPI. http://hg.python.org/cpython/rev/2b5cd6d4d149
New changeset e5a9755c967c by Antoine Pitrou in branch '3.3': Issue #12226: HTTPS is now used by default when connecting to PyPI. http://hg.python.org/cpython/rev/e5a9755c967c
New changeset 9839aa0e5b28 by Antoine Pitrou in branch 'default': Null merge (#12226 already fixed on default) http://hg.python.org/cpython/rev/9839aa0e5b28
Author: Antoine Pitrou (pitrou) *
Date: 2013-12-22 00:50
Closing as fixed, and opening a new issue for cert checking.
History
Date
User
Action
Args
2022-04-11 14:57:18
admin
set
github: 56435
2013-12-22 00:50:04
pitrou
set
status: open -> closed
resolution: fixed
messages: +
stage: resolved
2013-12-22 00:48:17
python-dev
set
messages: +
2013-12-22 00:36:00
python-dev
set
nosy: + python-dev
messages: +
2013-12-22 00:29:31
pitrou
set
assignee: eric.araujo ->
versions: - Python 2.6
2013-11-22 21:25:22
christian.heimes
set
messages: +
2013-11-22 21:10:23
pitrou
set
messages: +
2013-11-22 21:08:59
eric.araujo
set
messages: +
2013-11-10 19:01:57
techtonik
set
messages: +
2013-06-08 22:28:41
dstufft
set
messages: +
2013-06-08 22:26:19
techtonik
set
messages: +
2013-06-08 22🔞08
dstufft
set
messages: +
2013-06-08 21:46:04
techtonik
set
messages: +
2013-03-25 03:57:38
dstufft
set
nosy: + dstufft
messages: +
2013-03-25 03:23:42
benjamin.peterson
set
nosy: + richard
messages: +
2013-03-25 03:11:05
eric.araujo
set
messages: +
2013-03-24 12:31:53
benjamin.peterson
set
messages: +
2013-03-24 05:59:50
eric.araujo
set
messages: +
2013-03-18 23:44:20
Arfrever
set
messages: +
2013-02-24 11:52:57
Giovanni.Bajo
set
nosy: + Giovanni.Bajo
messages: +
2013-02-23 18:24:38
devin
set
nosy: + devin
2013-02-16 00:10:53
christian.heimes
set
nosy: + christian.heimes
messages: +
2013-02-04 22:20:25
eric.araujo
set
priority: release blocker -> high
assignee: tarek -> eric.araujo
2013-02-04 19:33:56
christian.heimes
set
dependencies: + Include CA bundle and provide access to system's CA
2013-02-04 17:10:56
christian.heimes
set
priority: normal -> release blocker
nosy: + larry, benjamin.peterson, georg.brandl
versions: + Python 3.4, - Python 3.1
2011-06-18 21:55:33
techtonik
set
messages: +
2011-06-18 21:29:40
eric.araujo
set
messages: +
2011-06-18 21:27:38
techtonik
set
messages: +
2011-06-18 20:54:59
eric.araujo
link
2011-06-18 20:54:59
eric.araujo
unlink
2011-06-18 20:54:25
eric.araujo
link
2011-06-18 20:52:38
eric.araujo
link
2011-06-18 20:52:20
eric.araujo
set
messages: +
2011-06-18 09:26:30
techtonik
set
messages: +
title: use secured channel for uploading packages to pypi -> use HTTPS by default for uploading packages to pypi
2011-06-07 11:16:05
pitrou
set
nosy: + pitrou
messages: +
2011-06-06 16:01:09
fdrake
set
messages: +
2011-06-06 15:50:56
eric.araujo
set
messages: +
2011-06-06 07:54:26
techtonik
set
messages: +
2011-06-04 14:33:46
eric.araujo
set
messages: +
2011-06-04 05:01:15
techtonik
set
messages: +
2011-06-03 22:59:20
loewis
set
messages: +
2011-06-03 20:52:44
techtonik
set
messages: +
2011-06-03 19:47:06
loewis
set
messages: +
2011-06-03 18:39:04
skrah
set
messages: +
2011-06-03 16:22:05
eric.araujo
set
messages: +
2011-06-01 15:41:35
jwilk
set
nosy: + jwilk
2011-06-01 15:22:52
tarek
set
messages: +
2011-06-01 15:17:47
fdrake
set
nosy: + fdrake
messages: +
2011-06-01 15:14:59
techtonik
set
messages: +
2011-06-01 15:11:11
techtonik
set
messages: +
2011-06-01 11:06:14
barry
set
messages: +
2011-06-01 07:30:14
skrah
set
nosy: + skrah
messages: +
2011-06-01 05:23:45
loewis
set
messages: +
2011-05-31 20:15:26
techtonik
set
messages: +
2011-05-31 19:13:19
tarek
set
nosy: + loewis, barry
messages: +
2011-05-31 18:24:25
Arfrever
set
nosy: + Arfrever
versions: + Python 3.2, Python 3.3
2011-05-31 16:54:23
brian.curtin
set
nosy: - brian.curtin
2011-05-31 16:54:08
techtonik
set
messages: +
2011-05-31 16:52:43
brian.curtin
set
nosy: + brian.curtin
messages: +
2011-05-31 16:51:54
brian.curtin
set
nosy: - brian.curtin
2011-05-31 16:51:45
brian.curtin
set
nosy: + brian.curtin
messages: +
2011-05-31 16:51:04
techtonik
create