[Python-Dev] Possible language summit topic: buildbots (original) (raw)

"Martin v. Löwis" martin at v.loewis.de
Sun Oct 25 10:47:10 CET 2009

Mark Dickinson wrote:

Would it be worth spending some time discussing the buildbot situation at the PyCon 2010 language summit? In the past, I've found the buildbots to be an incredibly valuable resource; especially when working with aspects of Python or C that tend to vary significantly from platform to platform (for me, this usually means floating-point, and platform math libraries, but there are surely many other things it applies to). But more recently there seem to have been some difficulties keeping a reasonable number of buildbots up and running. A secondary problem is that it can be awkward to debug some of the more obscure test failures on buildbots without having direct access to the machine. From conversations on IRC, I don't think I'm alone in wanting to find ways to make the buildbots more useful.

These are actually two issues: a) where do we get buildbot hardware and operators? b) how can we reasonably debug problems occurring on buildbots

For a), I think we can solve this only by redundancy, i.e. create more build slaves, hoping that a sufficient number would be up at any point in time.

So: what specific kinds of buildbots do you think are currently lacking? A call for volunteers will likely be answered quickly.

So the question is: how best to invest time and possibly money to improve the buildbot situation (and as a result, I hope, improve the quality of Python)?

I don't think money will really help (I'm skeptical in general that money helps in open source projects). As for time: "buildbot scales", meaning that the buildbot slave admins will all share the load, being responsible only for their own slaves.

On the master side: would you be interested in tracking slave admins?

What could be done to make maintenance of build slaves easier?

This is something that only the slave admins can answer. I don't think it's difficult - it's just that people are really unlikely to contribute to the same thing over a period of five years at a steady rate. So we need to make sure to find replacements when people drop out.

Or to encourage interested third parties to donate hardware and time?

Again: I think a call for volunteers would do (Steve, if you are reading this, please hold back just a few days before actually making such a call :-)

Are there good alternatives to Buildbot that might make a difference?

I think people have started working on such a thing. There are certainly alternatives; I'm fairly skeptical that they are good alternatives (but then, I'm the one who set up the buildbot installation in the first place).

What do other projects do?

I think that's really difficult to compare, since their testing often has a very different scope. I think CruiseControl is widely used.

These are probably the wrong questions; I'm hoping that a discussion would help produce the right questions, and possibly some answers.

I think these are good questions - just not for the summit. Setting up such a system is, conceptually, easy. It's also just a little work to set it up initially; the difficult part then is to keep it running (and no, a system where anybody can just post test results at any time without prior registration is still difficult to keep running).

The source of the problem is that such a system can degrade without anybody taking action. If the web server's hard disk breaks down, people panic and look for a solution quickly. If the source control is down, somebody will "volunteer" to fix it. If the automated build system produces results less useful, people will worry, but not take action.

Regards, Martin

More information about the Python-Dev mailing list