Issue 19740: test_asyncio problems on 32-bit Windows (original) (raw)

Created on 2013-11-23 22:13 by tim.peters, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
wait-for-handle.patch sbt,2013-11-24 13:56 review
Messages (9)
msg204116 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-11-23 22:13
With current default branch, test_asyncio always fails on my 32-bit Windows Vista box, in test_wait_for_handle: test test_asyncio failed -- Traceback (most recent call last): File "C:\Code\Python\lib\test\test_asyncio\test_windows_events.py", line 122, in test_wait_for_handle self.assertTrue(f.result()) AssertionError: False is not true If I comment out that line, the rest of the test passes. There's also a pile of annoying warnings: C:\Code\Python\lib\test\test_asyncio\test_events.py:195: ResourceWarning: unclosed <socket.socket fd=456, family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=0> gc.collect() C:\Code\Python\lib\test\test_asyncio\test_events.py:195: ResourceWarning: unclosed <socket.socket fd=200, family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=0> gc.collect() C:\Code\Python\lib\test\test_asyncio\test_events.py:195: ResourceWarning: unclosed <socket.socket fd=476, family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=0> gc.collect() C:\Code\Python\lib\test\test_asyncio\test_events.py:195: ResourceWarning: unclosed <socket.socket fd=508, family=AddressFamily.AF_INET6, type=SocketType.SOCK_STREAM, proto=0> gc.collect() C:\Code\Python\lib\test\test_asyncio\test_events.py:195: ResourceWarning: unclosed <socket.socket fd=528, family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 61433), raddr=('127.0.0.1', 61434)> gc.collect() C:\Code\Python\lib\test\test_asyncio\test_events.py:195: ResourceWarning: unclosed <socket.socket fd=516, fami ly=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 61434), raddr=('127.0.0.1', 61433)> gc.collect() C:\Code\Python\lib\test\test_asyncio\test_events.py:195: ResourceWarning: unclosed <socket.socket fd=556, family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=0> gc.collect() C:\Code\Python\lib\unittest\case.py:571: ResourceWarning: unclosed <socket.socket fd=544, family=AddressFamily .AF_INET, type=SocketType.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 61652), raddr=('127.0.0.1', 61651)> testMethod()
msg204130 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2013-11-23 23:39
It would be nice to try this on another Vista machine - the WinXP, Win7, Windows Server 2003 and Windows Server 2008 buildbots don't seem to show this failure. It looks as though the TimerOrWaitFired argument passed to the callback registered with RegisterWaitForSingleObject() is wrong. This might be fixable by doing an additional zero-timeout wait with WaitForSingleObject() to test whether the handle is signalled. (But this will prevent us from using wait_for_handle() with things like locks and semaphores where a succesful wait changes the state of the object represented by the handle.)
msg204140 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-11-24 00:44
@sbt, this is reproducible every time for me, so if there's anything you'd like me to try, let me know. I don't know anything about this code, and gave up after half an hour of trying to find out _where_ `False` was coming from - too convoluted for this old brain ;-)
msg204217 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2013-11-24 13:56
Could you try this patch?
msg204232 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-11-24 16:32
@sbt, success! With the patch, test_asyncio passed 10 times in a row on my box. Ship it :-)
msg204237 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-11-24 17:21
Possibly related: the successful test runs occurred running test_asyncio in isolation on a quiet machine. Then I fired off a full run of the test suite and used the machine for other things too. Then it failed: [ 23/387] test_asyncio ... various unclosed socket warnings ... test test_asyncio failed -- Traceback (most recent call last): File "C:\Code\Python\lib\test\test_asyncio\test_events.py", line 323, in test_reader_callback self.assertEqual(b''.join(bytes_read), b'abcdef') AssertionError: b'' != b'abcdef' ... 343 tests OK. 1 test failed: test_asyncio Unfortunately, this time it's hard to provoke :-(
msg204244 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-11-24 17:51
New changeset d71db7fe4872 by Richard Oudkerk in branch 'default': Issue #19740: Use WaitForSingleObject() instead of trusting TimerOrWaitFired. http://hg.python.org/cpython/rev/d71db7fe4872
msg204246 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2013-11-24 17:56
> Possibly related: ... That looks unrelated since it does not involve wait_for_handle(). Unfortunately test_utils.run_briefly() offers few guarantees when using the IOCP event loop.
msg204257 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2013-11-24 19:01
Agreed that that is probably unrelated. I suspect that all tests doing real I/O (stuff that goes through the OS kernel) and wait for it using run_briefly() are theoretically broken like that. It may just be harder to provoke for some tests than for others. The proper fix is to use test_utils.run_until(loop, , timeout) where is a lambda that computes whether the desired condition is reached. E.g. in this case we could use something like test_utils.run_until(self.loop, lambda: b''.join(bytes_read) == b'abcdef', 10) instead of the last five lines of the test (starting with the second run_briefly).
History
Date User Action Args
2022-04-11 14:57:54 admin set github: 63939
2013-11-24 19:02:35 gvanrossum set status: open -> closedtype: behaviorresolution: fixedcomponents: + Testsstage: resolved
2013-11-24 19:01:35 gvanrossum set messages: +
2013-11-24 17:56:55 sbt set messages: +
2013-11-24 17:51:55 python-dev set nosy: + python-devmessages: +
2013-11-24 17:21:37 tim.peters set messages: +
2013-11-24 16:32:03 tim.peters set messages: +
2013-11-24 13:56:07 sbt set files: + wait-for-handle.patchkeywords: + patchmessages: +
2013-11-24 00:44:12 tim.peters set messages: +
2013-11-23 23:39:33 sbt set messages: +
2013-11-23 22:15:44 pitrou set nosy: + sbt
2013-11-23 22:13:28 tim.peters create