[Python-Dev] Matrix testing against CPython releases (was: Re: Assertion in _PyManagedBuffer_FromObject()) (original) (raw)

Stefan Behnel stefan_ml at behnel.de
Sun Mar 4 13:49:45 CET 2012

Stefan Krah, 04.03.2012 12:33:

Thomas Wouters wrote:

Do you test against pydebug builds of Python, or otherwise a build that actually enables asserts? Yes, I do (and much more than that): http://hg.python.org/features/cdecimal/file/40917e4b51aa/Modules/decimal/python/runall-memorydebugger.sh http://hg.python.org/features/cdecimal/file/40917e4b51aa/Modules/decimal/python/runall.bat It's automated, so it's not a big deal. You get 100% coverage, with and without threads, all machine configurations, pydebug, refleaks, release build and release build with Valgrind.

Same for Cython. We continuously test against the debug builds of all CPython branches since 2.4 (the oldest we support), as well as the latest developer branch, using both our own test suite and Python's regression test suite.

https://sage.math.washington.edu:8091/hudson/

https://sage.math.washington.edu:8091/hudson/view/python/

BTW, I can warmly recommend Jenkins' matrix builds for this kind of compatibility testing. Here's an example:

https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/

Basically, you write a build script and Jenkins configures it using environment variables that define the specific setup, e.g. Python 2.7 with C backend. It'll then run all combinations in parallel (optionally filtering out nonsense combinations or preferring combinations that should fail the build early) and present the results both as an aggregated view and in separate per-setup views. It also uses file hashes to remember where the dependencies came from, e.g. which build created the CPython installation that was used for testing, so that you can jump right to the build log of the dependency to check for relevant changes that may have triggered a test failure. Oh, and you can just copy such a job config to set up a separate set of test jobs for a developer's branch, for example. A huge help in distributed developer settings, or when you want to get a GSoC student up and running.

Stefan

More information about the Python-Dev mailing list