| msg103590 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2010-04-19 12:42 |
| If an URL doesn't answer, the whole test hung. Many buildbots turned red because an URL (maybe ftp://ftp.kernel.org/pub/linux/kernel/README) didn't answer during few minutes (it works again). We should add a timeout, eg. 5 minutes. I don't know what to do on timeout: ignore the test? skip the test? Ignore should be fine. |
|
|
| msg103608 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2010-04-19 15:00 |
| Attached patch sets the timeout to 60 seconds. It writes "<timeout: %s>" % url to stderr on timeout. I don't know better place to write the error. (The test doesn't fail, it just continues to the next URL) |
|
|
| msg103625 - (view) |
Author: Senthil Kumaran (orsenthil) * |
Date: 2010-04-19 19:08 |
| Hello Victor, The patch looks for fine for a timeout enforcement. A couple of changes required. - if isinstance(err[0], timeout): + if isinstance(err[0], socket.timeout): And, in this portion: + try: + with test_support.transient_internet(): + buf = f.read() + except socket.timeout: + print >>sys.stderr, "<timeout: %s>" % url f.close() debug("read %d bytes" % len(buf)) The last line references, buf. It wont be available if a socket.timeout had happened. So, it can simply be moved to the line below buf = f.read(). Attaching the patch with these changes. Question: Were you able to simulate a timeout at kernel.org site to test this? If yes, how? I am unable to simulate (by changing the kernel.org:81, kernel42.org etc), it immediately throws a connection refused. ( In general, I could simulate the timeout by urlopen('www.google.com:81', timeout=30) and test the exception) As build-bot seems to be hung, a timeout enforcement as the patch accomplishes is fine. |
|
|
| msg103659 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2010-04-19 23:23 |
| > The last line references, buf. It wont be available if a > socket.timeout had happened. So, it can simply be moved to > the line below buf = f.read(). Attaching the patch with > these changes. Good catch :-) > Question: Were you able to simulate a timeout at kernel.org > site to test this? If yes, how? I used a firewall rule. On Linux, you can try: iptables -I OUTPUT DROP to drop all output packets (and then iptables -F to remove all rules). It's not reliable, but the test does sometimes fail :-) |
|
|
| msg103678 - (view) |
Author: Senthil Kumaran (orsenthil) * |
Date: 2010-04-20 06:56 |
| Committed revision 80236 in the trunk. The build-bots should not hang now. Shall merge to other branches too. |
|
|
| msg103689 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2010-04-20 09:30 |
| Don't you port the fix to py3k? |
|
|
| msg103690 - (view) |
Author: Senthil Kumaran (orsenthil) * |
Date: 2010-04-20 09:55 |
| Yes, definitely it needs to be ported. I realized that, py3k was in a bad shape at moment. File "Lib/test/test_urllib2net.py", line 151, in _test_urls import logging File "/home/senthil/python/py3k/Lib/logging/__init__.py", line 43, in import threading File "/home/senthil/python/py3k/Lib/threading.py", line 34, in TIMEOUT_MAX = _thread.TIMEOUT_MAX AttributeError: 'module' object has no attribute 'TIMEOUT_MAX' This is without the patch in. I was looking at what was wrong there, and once that is fix, merging this change to py3k would be fine. |
|
|
| msg103692 - (view) |
Author: Senthil Kumaran (orsenthil) * |
Date: 2010-04-20 10:48 |
| make distclean erased the problem. :) merged to py3k in r80256 and release31-maint in r80257. Had merged to release26-maint in r80237 |
|
|