[Python-Dev] Automatic installer builds (was Re: Fwd: Broken link to download (Mac OS X)) (original) (raw)

David Bolen db3l.net at gmail.com
Sat Apr 17 22:19:51 CEST 2010

"Martin v. Löwis" <martin at v.loewis.de> writes:

This actually happened on Windows - some people now recommend to run the buildbot scripts on a regular developer checkout, because they supposedly do the right things.

I have to admit that I'm guilty of this (though to be fair most of my test builds are buildbot-related), if only because it takes care of all the external stuff automatically and self-contained in the build tree.

I would rather prefer to have the buildbot run the same commands that we recommend developers to run. The knowledge about these should be in the README, and it should be stable knowledge, i.e. require infrequent changes. This is true for Unix: configure/make/make test/make clean had been the correct procedure for ten years now. The Unix buildbot only deviates slightly, to have the slaves run a more expensive version of the test suite.

In digging around a bit, it would appear that there's actually quite a bit of OSX support already in the Makefile (either the main one or the one in Mac). There's even a target that tests both halves of a universal build (using rosetta for the PPC version) on an Intel box.

I suspect it's just a question of setting up a Mac-appropriate builder, using the universal SDK in the configure step, and whatever Makefile targets are deemed best and current, perhaps with the addition of support for pulling in the third party stuff through externals or whatever. A first pass could just be to factor that process into a separate stage of build-installer that could be run independently of the rest of the installer build process.

In the interim, I've generated the third-party libraries via the current trunk build-installer script and installed them in /usr/local on my buildbot, so they are found by any of the buildbot builds using the stock configure/make approach. Other than a slight update to the bzip version, looks like the dependency versions in the installer script haven't changed for over a year, so I suspect the libraries are fine for any of the branches currently being built.

I also updated to the latest 8.4.19 Tcl/Tk in /Library/Frameworks since I saw some interpreter crashes in tests in what appeared to be a Tcl code path. It had been building against my system 8.4.7 Tcl. The Windows buildbot uses Tcl 8.5 - not sure if that should be preferred for the Mac buildbot as well, but will leave it at 8.4 for now.

I think this should create test builds more representative of the final binaries. It's not testing a universal framework build, but the test target only tests the Intel path anyway, the generated code should still be the same, and the same libraries are in use.

If anyone more familiar with this side of things for OSX has some spare time down the road, I'd be happy to help work on improving the process for the buildbot.

I'd be interested in setting up daily builds then. For these, I think it's fine that they may differ slightly from the official binaries - people would recognize that they are testing a different set of binaries.

We can certainly start by just trying to automate the execution of build-installer, something I suspect can be completely controlled from the master side, and then uploading the resulting dmg file. I'd be happy to help coordinate any experiments offline if you're interested.

I do currently have a DMG built for 2.7 Beta 1, if it would be useful.

-- David

More information about the Python-Dev mailing list