[Python-Dev] How to watch buildbots? (original) (raw)
Victor Stinner vstinner at redhat.com
Wed May 30 10:06:39 EDT 2018
- Previous message (by thread): [Python-Dev] How to watch buildbots?
- Next message (by thread): [Python-Dev] Withdraw PEP 546? Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
2018-05-30 14:30 GMT+02:00 Ivan Pozdeev via Python-Dev <python-dev at python.org>:
What's the big idea of separate buildbots anyway? I thought the purpose of CI is to test everything before it breaks the main codebase. Then it's the job of the contributor rather than maintainer to fix any breakages.
I will answer more generally.
Technically, buildbots support to send emails to author of changes which introduced a regression.
But a build may test a single change or dozens of new changes.
Moreover, our test suite is not perfect: they are at least 5 known tests which fail randomly. Even if we fix these unstable tests, it's also "common" that buildbots fail for "external" reasons:
network failure: fail to clone the GitHub repository
functional test using an external service and the service is down. I started to list external services used by "unit" tests: http://vstinner.readthedocs.io/cpython.html#services-used-by-unit-tests
vacuum cleaner: https://mail.python.org/pipermail/python-buildbots/2017-June/000122.html
many other random reasons...
Since two years, I'm trying to fix all tests failing randomly, but as I just explained, it's really hard to get a failure rate of 0%.
I'm not sure that we can "require" authors of pull requests to understand buildbot failures...
So I prefer to keep the status quo: filter buildbot failures manually.
Victor
- Previous message (by thread): [Python-Dev] How to watch buildbots?
- Next message (by thread): [Python-Dev] Withdraw PEP 546? Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]