[Python-Dev] buildbot (original) (raw)

Brian Warner warner at lothar.com
Wed Jan 4 09:30:58 CET 2006

Currently, branches are not supported, because buildbot is somewhat limited. When I get a solution for this problem, I plan to have all buildbots build both the trunk and the 2.4 branch; such builds would only be initiated whenever something is committed on the branch.

Branch support appeared fairly recently (buildbot-0.7.0, about two months ago), and I believe it should be sufficient to trigger builds for both the trunk and the 2.4 branch. The last part of the setup involves modifying the svn_buildbot.py hook script, though, so I won't claim this support is complete yet.

The basic idea is that you create multiple Scheduler instances, one for each branch you want to monitor. Both Schedulers are configured to trigger all the Builders. You configure the SVN-checkout step of the build process with a baseURL= argument to provide the common portion of the SVN URL, and add a defaultBranch= argument to point to the trunk. Each Build is done for a specific branch, and the SVN checkout step just appends the branch name to the baseURL to figure out what to pass to 'svn co %s'

Assuming the ChangeSource is providing branch information properly, this will result in any change to the trunk causing a trunk build, and any change to the 2.4 branch causing a 2.4 build.

Your master.cfg file will probably want to have some of the following elements:

---BEGIN--- from buildbot.changes.pb import PBChangeSource c['sources'] = PBChangeSource() # N.B.: don't set prefix=

from buildbot.scheduler import Scheduler all_builders=["sparc solaris10 gcc trunk", "g5 osx trunk", "amd64 gentoo trunk", "x86 gentoo trunk"] s1 = Scheduler("trunk scheduler", "trunk", 260, all_builders) s2 = Scheduler("2.4 scheduler", "branches/release24-maint", 260, all_builders) c['schedulers'] = [s1, s2]

... from buildbot.process.factory import s from buildbot.process.step import SVN checkoutstep = s(SVN, baseURL="http://svn.python.org/projects/python/", defaultBranch="trunk", mode="copy") steps = [checkoutstep, ...]

---END---

The tricky part is getting branch-annotated Changes into the buildbot in the first place. I assume you're using the svn_buildbot.py script. The version provided in the current release always sets the branch to 'None', which implies the default branch. You will have to edit this script to give it the ability to decide which branch each change is for (by adding some code which does a test like filename.startswith("trunk/"), perhaps), then include the branch name in the RPC call that informs the buildmaster about the new Change. Basically you'll need to change this call:

    return persp.callRemote('addChange', {'who': who,
                                          'files': changed,
                                          'comments': message,
                                          'revision': revision})

into one like:

    branchname = "trunk" # or "branches/release24-maint"
    return persp.callRemote('addChange', {'who': who,
                                          'branch': branchname,
                                          'files': changed,
                                          'comments': message,
                                          'revision': revision})

There's been some discussion on the buildbot-devel list about the best way to implement branch-determination. One school of thought says it belongs close to the repository (hence inside a hook script like svn_buildbot.py). Another says it is easier to configure from the buildbot side, in which case you'd probably want to modify buildbot.changes.pb.PBChangeSource to analyze the filenames and decide which branch they live on there.

There was also a patch1 to add some regexps to svn_buildbot.py to do this kind of branch determination. I haven't been able to review it properly yet, but it may give you some ideas. svn_buildbot.py certainly needs help here. The rest of the Buildbot is branch-enabled and ready to go. The only lingering issues are with status delivery: the main HTML "Waterfall" HTML will interleave builds of both branches on the same page, which could be a bit confusing (if the top line is red, does that mean the trunk is broken, or the 2.4 branch?). Fixing this (by creating separate pages for each branch) is high on the TODO list.

If there's any way I can help out further, please let me know. I'm ecstatic that Python is using a buildbot.. if you have any feature suggestions or bug reports, please send them to me, to the buildbot-devel mailing list2, or file them in the sf.net bugtracker3. Or corner me at PyCon, or in #twisted (where I can sometimes be found as ).

thanks! -Brian (author of Buildbot)

More information about the Python-Dev mailing list