[Python-Dev] Buildbots and regrtest timeout (original) (raw)

Victor Stinner victor.stinner at haypocalc.com
Mon Jun 6 12:29:46 CEST 2011

Hi,

Stephan Krah asked me to change how the default timeout is defined for regrtest (issue #12250):

"The implicit timeout in regrtest.py makes it harder to write automated test scripts for 3rd party modules. First, you have to remember to set --timeout=0 for long running tests. Then, you have to remember not to use the --timeout option when compiling --without-threads.

I'd much prefer that there's no timeout unless explicitly specified. For the buildbots, I think this could be done in the Makefile."

First I replaced the hardcoded constant in regrtest.py by a command line argument (--timeout=3600) in the TESTOPTS variable of the Makefile, so if you call regrtest directly (without make), there is no more default timeout.

But today I saw a a buildbot timeout without any traceback: a possible hang in test_io on "x86 FreeBSD 7.2 3.x" buildbot, "command timed out: 3900 seconds without output". I realized that some buildbots (all buildbots?) override the TESTOPTS variable ("make buildbottest TESTOPTS= TESTPYTHONOPTS=" for "x86 FreeBSD 7.2 3.x").

I moved the timeout option from TESTOPTS to a new variable: TESTTIMEOUT, with a default value of 1 hour (3600 seconds). The timeout is now only used for "make buildbottest". Use TESTTIMEOUT=0 to disable completly regrtest timeout. The timeout argument if ignored (with a warning) if faulthandler.dump_tracebacks_later() function is missing (e.g. if Python is compiled without threads).

Please, don't disable the TESTTIMEOUT option for your buildbot. You may want to use a shorter or longer timeout, it just have to be smaller than the buildbot timeout (3900 seconds by default, 1 hour 6 minutes, just a little bit more than regrtest timeout: 1 hour). If you want a timeout longer than 1 hour, you can to change TESTTIMEOUT and the buildbot timeout (I don't know how to configure the buildbot timeout).

I didn't touch test, testall, testuniversal, quicktest make rules, which don't use the regrtest timeout anymore. I would prefer to use the same timeout options for all test rules in TESTOPTS, but it doesn't work because some buildbots do override TESTOPTS variable.

Victor

More information about the Python-Dev mailing list