Issue 5798: test_asynchat fails on Mac OSX (original) (raw)

Created on 2009-04-20 11:02 by donmez, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (40)

msg86193 - (view)

Author: Ismail Donmez (donmez) *

Date: 2009-04-20 11:02

Using latest python 2.6 branch; test_asynchat fails with the following error:

error: uncaptured python exception, closing channel <test.test_asynchat.echo_client at 0x1689238> (<class 'socket.error'>:[Errno 9] Bad file descriptor [/Users/cartman/Sources/python-2.6/Lib/asyncore.py|readwrite|107] [/Users/cartman/Sources/python-2.6/Lib/asyncore.py|handle_expt_event|441] [|getsockopt|1] [/Users/cartman/Sources/python-2.6/Lib/socket.py|_dummy|165])

Using Mac OSX 10.5.6.

msg86529 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2009-04-25 16:25

I'm seeing this failure on 2.7, 3.0 and 3.1 as well.

msg87368 - (view)

Author: Ismail Donmez (donmez) *

Date: 2009-05-07 11:42

Still fails in 3.1 beta1.

msg87371 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2009-05-07 12:09

I tried to track this down, but ran out of time. Here's the little that I discovered; maybe someone else with access to OS X (which I have) and an understanding of poll (which I lack) can build on this.

The failure has to do with select.poll differences between Linux and OS X. It seems that OS X can return POLLIN | POLLPRI | POLLHUP in situations where Linux just returns POLLIN, and this combination of flags causes the code in asyncore to first close the socket and then try to read from it. The relevant code is at around line 440 of Lib/asyncore.py.

msg87375 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2009-05-07 16:50

r56632 looks relevant:

"""When running asynchat tests on OS X (darwin), the test client now overrides asyncore.dispatcher.handle_expt to do nothing, since select.poll gives a POLLHUP error at the completion of these tests. Added timeout & count arguments to several asyncore.loop calls to avoid the possibility of a test hanging up a build. [GSoC - Alan McIntyre]"""

Adding Facundo Batista and Alan McIntyre to the nosy list. Does either of you know what's going on with this issue?

msg87377 - (view)

Author: Jean Brouwers (MrJean1)

Date: 2009-05-07 17:01

Here is a (new?) failure of test_asynchat with Python 3.1b1 on MacOS X 10.4.11 (Intel).

% make test .... test test_asynchat produced unexpected output:

*** lines 2-16 of actual output doesn't appear in expected output after line 1:

....

% ./python.exe Lib/test/test_asynchat.py test_close_when_done (main.TestAsynchat) ... ok test_empty_line (main.TestAsynchat) ... ok test_line_terminator1 (main.TestAsynchat) ... ok test_line_terminator2 (main.TestAsynchat) ... ok test_line_terminator3 (main.TestAsynchat) ... ok test_none_terminator (main.TestAsynchat) ... ok test_numeric_terminator1 (main.TestAsynchat) ... ok test_numeric_terminator2 (main.TestAsynchat) ... ok test_simple_producer (main.TestAsynchat) ... ok test_string_producer (main.TestAsynchat) ... ok test_close_when_done (main.TestAsynchat_WithPoll) ... ok test_empty_line (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x107cf30> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1b1/Lib/asyncore.py|readwrite|106] [../Python- 3.1b1/Lib/asyncore.py|handle_expt_event|440]) ok test_line_terminator1 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x107c3f0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python- 3.1b1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x107ceb0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python-3.1b1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x107cf50> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python-3.1b1/Lib/asyncore.py|handle_expt_event|440]) ok test_line_terminator2 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x107ce70> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python- 3.1b1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x107ced0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python-3.1b1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x107cf30> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python-3.1b1/Lib/asyncore.py|handle_expt_event|440]) ok test_line_terminator3 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x107ceb0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python- 3.1b1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x107cf50> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python-3.1b1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x107cef0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python-3.1b1/Lib/asyncore.py|handle_expt_event|440]) ok test_none_terminator (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x107cf50> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python- 3.1b1/Lib/asyncore.py|handle_expt_event|440]) ok test_numeric_terminator1 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x107ce10> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python- 3.1b1/Lib/asyncore.py|handle_expt_event|440]) ok test_numeric_terminator2 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x107ce50> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python- 3.1b1/Lib/asyncore.py|handle_expt_event|440]) ok test_simple_producer (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x107cdb0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python- 3.1b1/Lib/asyncore.py|handle_expt_event|440]) ok test_string_producer (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x107cf50> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1b1/Lib/asyncore.py|readwrite|106] [../Python- 3.1b1/Lib/asyncore.py|handle_expt_event|440]) ok test_find_prefix_at_end (main.TestHelperFunctions) ... ok test_basic (main.TestFifo) ... ok test_given_list (main.TestFifo) ... ok

Ran 23 tests in 7.712s

OK

msg87379 - (view)

Author: R. David Murray (r.david.murray) * (Python committer)

Date: 2009-05-07 17:27

See also issue 1161031, especially Giampaolo's suggestion near the end. Seems like it might be relevant. I was the one who merged Josiah's trunk fix into othe other branches, to get things back into sync, but I don't claim to understand the code at any deep level. From the review I did at my level of knowledge it seems like the change to the behavior of when handle_expt_event is called is correct, but I was a little worried about backward compatibility. I now wonder if that fix should be backed out of 2.6 and 3.0, and advertised in What's New for 2.7 and 3.1.

msg87386 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-05-07 18:30

Looking at trunk, it seems like one reasonable option is to swap the order of handle_close() and handle_expt_event() testing and calls. That would keep all reading/writing before handle_close(), which should be correct.

msg87392 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2009-05-07 19:10

Josiah, that solution isn't working for me; it looks as though there's a deeper weirdness: what I'm seeing is that on OS X, in e.g. test_emptyline, we end up calling the readwrite function in asyncore.py with flags = POLLIN | POLLPRI | POLLHUP. The first thing that gets called is obj.handle.read_event, and bizarrely it seems to be the read_event call that has the effect of closing the socket. So the socket's already closed by the time we get to obj.handle_expt_event and the same error as before occurs.

I'm still trying to figure out why.

msg87393 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2009-05-07 19:28

So the sequence of events seems to be: asyncore.readwrite calls obj.handle_read_event ... which calls obj.handle_read (3rd branch of handle_read_event) ... which is defined in asynchat.py; it calls obj.recv ... (back in asyncore now): recv calls obj.socket.recv ... and gets no data, so it then calls obj.handle_close ... which calls obj.close(), and so closes obj.socket as well.

and now obj.handle_expt_event gets confused because the socket is closed.

Now what?

msg87400 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-05-07 19:52

It would seem that we need to be more defensive in our calls. We need to check to make sure that the socket isn't closed before calling read/write/expt events.

msg87402 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-05-07 20:03

Mark, try this:

    if flags & select.POLLIN and (obj.connected or obj.accepting):
        obj.handle_read_event()
    if flags & select.POLLOUT and obj.connected:
        obj.handle_write_event()
    if flags & select.POLLPRI and obj.connected:
        obj.handle_expt_event()
    if flags & (select.POLLHUP | select.POLLERR | select.POLLNVAL):
        obj.handle_close()

msg87403 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2009-05-07 20:10

That fixes test_empty_line (in test_asynchat.py), but now I get a hang in test_close_when_done.

By the way, here's an interesting site that goes some way to explaining what to expect from select.poll and EOF.

http://www.greenend.org.uk/rjk/2001/06/poll.html

msg87404 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-05-07 20:15

Try getting rid of the "and" clause in the select.POLLIN .

msg87405 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-05-07 20:18

To be clear, make the first test read... if flags & select.POLLIN: obj.handle_read_event()

msg87406 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2009-05-07 20:30

Getting rid of the and ... on the handle_read_event didn't make a difference: I still get the hang in test_close_when_done.

But if I get rid of the and clause on the handle_write_event branch then all the test_asynchat tests pass. So that block now looks like:

    if flags & select.POLLIN and (obj.connected or obj.accepting):
        obj.handle_read_event()
    if flags & select.POLLOUT:
        obj.handle_write_event()
    if flags & select.POLLPRI and obj.connected:
        obj.handle_expt_event()
    if flags & (select.POLLHUP | select.POLLERR | select.POLLNVAL):
        obj.handle_close()

in my code. So we're making progress (maybe). Unfortunately, test_asyncore now fails with:

====================================================================== FAIL: test_readwrite (main.HelperFunctionTests)

Traceback (most recent call last): File "Lib/test/test_asyncore.py", line 161, in test_readwrite self.assertEqual(getattr(tobj, attr), attr==expectedattr) AssertionError: False != True

msg87408 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-05-07 21:20

I went ahead and plugged my mac in (which reminded me of why I unplugged it in the first place), and I'm able to reproduce your error in the test after my proposed modifications.

One thing to remember is that the test is broken; we rely on a .connected attribute, which the test objects lack.

The attached patch fixes the test on my OS X machine, which you should test.

msg87416 - (view)

Author: Jean Brouwers (MrJean1)

Date: 2009-05-08 00:11

I reran the test_synchat.py test after patching to the Lib/asyncore.py and Lib/test/test-asynchat.py files of Python-3.1b1 on MacOS X 10.4.11 (Intel).

The test now hangs in ... test_string_producer (main.TestAsynchat) ... ok test_close_when_done (main.TestAsynchat_WithPoll) ...

The test also hangs, after removing the patches from the Lib/test/test_asynchat.py file but keeping the patched Lib/asyncore.py file.

msg87417 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-05-08 00:18

As an aside, I was testing against trunk, not 3.1b1 .

msg87419 - (view)

Author: Jean Brouwers (MrJean1)

Date: 2009-05-08 00:40

Understood.

On Thu, May 7, 2009 at 5:18 PM, Josiah Carlson <report@bugs.python.org>wrote:

Josiah Carlson <josiahcarlson@users.sourceforge.net> added the comment:

As an aside, I was testing against trunk, not 3.1b1 .

Python tracker <report@bugs.python.org> <http://bugs.python.org/issue5798>

msg87444 - (view)

Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer)

Date: 2009-05-08 15:15

Has anyone already tried to modify readwrite() as I was suggesting here? http://bugs.python.org/issue1161031#msg84892

msg87447 - (view)

Author: Jean Brouwers (MrJean1)

Date: 2009-05-08 16:18

Indeed, the unmodified test_asynchat.py now passes in Python 3.1b1. The readwrite function in asyncore looks like this now:

def readwrite(obj, flags): try: if flags & (select.POLLHUP | select.POLLERR | select.POLLNVAL): obj.handle_close() else: if flags & select.POLLIN: obj.handle_read_event() if flags & select.POLLOUT: # and obj.connected: obj.handle_write_event() if flags & select.POLLPRI: # and obj.connected: obj.handle_expt_event() except _reraised_exceptions: raise except: obj.handle_error()

msg87453 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-05-08 17:49

One of the issues with using the method that Giampaolo describes, which I explained to him, is that generally if someone sends you data, you want to receive it. You can get both data and the signal that someone disconnected, in particular, with asynchat's .close_when_done() method.

If you get a read signal, generally you want to try to pull more data before you close, because when you close, you can never get any pending data again.

For example, if I were to use: chat.send("commit\n") chat.close_when_done() It's very likely that on even moderate latency network connections (10ms), the other end would get the close signal at the same time as the read for the "commit\n" message. But since they close before they read, they never get the "commit" message, and maybe a transaction is reverted.

My primary concern is keeping the library correct. Adding attributes in the test cases don't bother me.

Jean: on what platform are you experiencing the hang? Are you using trunk or 3.1?

msg87455 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2009-05-08 18:24

With Josiah's patch, on trunk (r72475), test_asyncore passes, but test_asynchat is still hanging in the test:

test_close_when_done (main.TestAsynchat_WithPoll) ...

The hang occurs at the line "data = conn.recv(1)" in the run method of the echo_server class in Lib/test/test_asynchat.py.

msg87458 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2009-05-08 18:53

With Giampaolo's suggested change, all tests pass. My understanding of what asyncore/asynchat is intended to do and how it might be used in practice is weak, so I don't feel qualified to comment on the correctness of the change.

One of the issues with using the method that Giampaolo describes, which I explained to him, is that generally if someone sends you data, you want to receive it. You can get both data and the signal that someone disconnected, in particular, with asynchat's .close_when_done() method.

Is it possible to add a test for this sort of situation, to make sure that future changes take it into account?

msg87465 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-05-08 19:38

Ok, so I was running "test_asyncore" and not "test_asynchat".

The issue on OS X is that when a new socket is connecting, select.poll() shows the socket as being writable when it first connects, but using my variant, .connected is False while it's waiting for the connection.

Options: If we swap the pollin/pollout conditions, that would fix the hang on mac, but cause the hang on others (I'm not really worried about not doing a write after a read, .push() in asynchat performs an immediate send).

If we get rid of the .connected check, then when a .read() fails and causes a close, then we get an exception in handle_write().

If we swap to Giampaolo's variant, then we could lose data that was legitimately sent by a client.

I've got a version that doesn't hang on OS X, and should work on all platforms. It's ugly, but it works.

msg87467 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2009-05-08 20:06

asyncore_fix_mac_2.patch fixes all test_failures for me, on both trunk and py3k.

msg87470 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-05-08 20:57

Here's an option that doesn't use .connected, which some people have expressed distaste for.

def readwrite(obj, flags): try: if flags & select.POLLIN: obj.handle_read_event() if flags & select.POLLOUT: obj.handle_write_event() if flags & select.POLLPRI: obj.handle_expt_event() if flags & (select.POLLHUP | select.POLLERR | select.POLLNVAL): obj.handle_close() except socket.error, e: if e.args[0] not in (EBADF, ECONNRESET, ENOTCONN, ESHUTDOWN, ECONNABORTED): obj.handle_error() else: obj.handle_close() except _reraised_exceptions: raise except: obj.handle_error()

It works on OS X and should work just as well on other platforms.

msg87477 - (view)

Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer)

Date: 2009-05-08 22:20

Even if that patch would fix this issue (and I'm pretty sure it does) I don't think it's a good idea handling all those errors (EBADF, ECONNRESET, ENOTCONN, ESHUTDOWN, ECONNABORTED) in readwrite() function.

Although those error codes unmistakably remark a disconnect event condition (EBADF too? are we sure it always means that?) I would prefer that be done in asyncore.dispatcher() and handle_close() be called from there.

That's the implementation asyncore has always used until now where the poller logic (poll(), poll2()) has always been kept separated from the connection logic (asyncore.dispatcher) where the "if socket.err[0] in ..." stuff is done.

Unfortunately I haven't got an OS X box to test against and verify by myself, but If I'm not mistaken by looking at the traceback messages pasted by other folks, it seems that the exception takes place in handle_expt_event() at this point:

err = self.socket.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR)

If that's the case I think it makes more sense catching EBADF from there rather than breaking the logic used so far, also because, if I'm not mistaken, the problem has been introduced by this very line which has been added in Python 2.6.

msg88798 - (view)

Author: Ned Deily (ned.deily) * (Python committer)

Date: 2009-06-03 07:41

Still happening on 3.1rc1.

Should this be considered a release blocker for 3.1?

msg88813 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-06-03 15:03

If it's failing, and asyncore is still in 3.1, then I would argue yes.

I'll submit a fix to trunk and 3.1 based on my version below (unless anyone has any outstanding concerns, or believes that it doesn't work for them).

msg88822 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-06-03 19:49

Fixed in trunk in 73182, fixed in py3k in 73183.

Closing as fixed.

msg88855 - (view)

Author: Ned Deily (ned.deily) * (Python committer)

Date: 2009-06-04 00:06

Verified test_asynchat no longer fails in trunk nor py3k. Thanks!

msg88888 - (view)

Author: Jean Brouwers (MrJean1)

Date: 2009-06-04 17:08

Well, with fresh build of Python 3.1rc1 on MacOS X 10.4.11 Tiger (Intel) test_asynchat.py rev 73183 still seems to fail, perhaps differently. Here is 3 different results. First, rev 73183:

% ./python.exe Lib/test/test_asynchat73183.py test_close_when_done (main.TestAsynchat) ... ok test_empty_line (main.TestAsynchat) ... ok test_line_terminator1 (main.TestAsynchat) ... ok test_line_terminator2 (main.TestAsynchat) ... ok test_line_terminator3 (main.TestAsynchat) ... ok test_none_terminator (main.TestAsynchat) ... ok test_numeric_terminator1 (main.TestAsynchat) ... ok test_numeric_terminator2 (main.TestAsynchat) ... ok test_simple_producer (main.TestAsynchat) ... ok test_string_producer (main.TestAsynchat) ... ok test_close_when_done (main.TestAsynchat_WithPoll) ... ok test_empty_line (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x10835b0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_line_terminator1 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083550> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083610> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x10835f0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_line_terminator2 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083630> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083870> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x10838b0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_line_terminator3 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083610> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083550> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x10835f0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_none_terminator (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083550> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_numeric_terminator1 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083630> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_numeric_terminator2 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x10837f0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_simple_producer (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083910> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_string_producer (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x10835b0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_find_prefix_at_end (main.TestHelperFunctions) ... ok test_basic (main.TestFifo) ... ok test_given_list (main.TestFifo) ... ok

Ran 23 tests in 7.717s

OK

Second, this is the result of test_asyncat.py included in Python 3.1rc1:

% ./python.exe Lib/test/test_asynchat.py test_close_when_done (main.TestAsynchat) ... ok test_empty_line (main.TestAsynchat) ... ok test_line_terminator1 (main.TestAsynchat) ... ok test_line_terminator2 (main.TestAsynchat) ... ok test_line_terminator3 (main.TestAsynchat) ... ok test_none_terminator (main.TestAsynchat) ... ok test_numeric_terminator1 (main.TestAsynchat) ... ok test_numeric_terminator2 (main.TestAsynchat) ... ok test_simple_producer (main.TestAsynchat) ... ok test_string_producer (main.TestAsynchat) ... ok test_close_when_done (main.TestAsynchat_WithPoll) ... ok test_empty_line (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x10835b0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_line_terminator1 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083550> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083610> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x10835f0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_line_terminator2 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083630> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083870> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x10838b0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_line_terminator3 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083610> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083550> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) error: uncaptured python exception, closing channel <__main__.echo_client at 0x10835f0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python-3.1rc1/Lib/asyncore.py|readwrite|106] [../Python- 3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_none_terminator (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083550> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_numeric_terminator1 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083670> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_numeric_terminator2 (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x10837f0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_simple_producer (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x1083910> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_string_producer (main.TestAsynchat_WithPoll) ... error: uncaptured python exception, closing channel <__main__.echo_client at 0x10835b0> (<class 'socket.error'>:[Errno 9] Bad file descriptor [../Python- 3.1rc1/Lib/asyncore.py|readwrite|106] [../Python-3.1rc1/Lib/asyncore.py|handle_expt_event|440]) ok test_find_prefix_at_end (main.TestHelperFunctions) ... ok test_basic (main.TestFifo) ... ok test_given_list (main.TestFifo) ... ok

Ran 23 tests in 7.721s

OK

Both are not the exact same, but equivalent. Lastly, here are 4 snippets from the 'make test' output.

........

test test_asynchat produced unexpected output:

*** lines 2-16 of actual output doesn't appear in expected output after line 1:

........

test_cmd_line test test_cmd_line failed -- Traceback (most recent call last): File "../Tools/Python-3.1rc1/Lib/test/test_cmd_line.py", line 145, in test_run_code 0) AssertionError: 1 != 0 ........

test test_smtplib produced unexpected output:

*** line 2 of actual output doesn't appear in expected output after line 1:

........

308 tests OK. 3 tests failed: test_asynchat test_cmd_line test_smtplib 25 tests skipped: test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses test_dbm_gnu test_epoll test_largefile test_nis test_normalization test_ossaudiodev test_pep277 test_smtpnet test_socketserver test_startfile test_timeout test_tk test_ttk_guionly test_urllib2net test_urllibnet test_winreg test_winsound test_xmlrpc_net test_zipfile64 Those skips are all expected on darwin. make: *** [test] Error 1

msg88889 - (view)

Author: R. David Murray (r.david.murray) * (Python committer)

Date: 2009-06-04 17:19

I presume you mean "a fresh build of 3.1rc1+" (from svn)? 3.1rc1 does not contain the most recent fix.

msg88890 - (view)

Author: Josiah Carlson (josiahcarlson) * (Python triager)

Date: 2009-06-04 18:01

I installed 3.1rc1 on my OS X (10.5.?) machine, updated asynchat, and ran the test with and without my change. Without my change, it breaks in the way described numerous times. With my change, it seems to work fine on OS X, linux, and Windows for me.

Looking at line 106 in py3k/Lib/asyncore.py (http://svn.python.org/view/python/branches/py3k/Lib/asyncore.py? annotate=73183) gets me "obj.handle_close()". Your tracebacks show line 106 calling "obj.handle_expt_event()", which is the old code.

msg88920 - (view)

Author: Jean Brouwers (MrJean1)

Date: 2009-06-05 04:18

Correct. With new Lib/asyncore.py file rev 73183 all 23 tests in the original test_asynchat.py pass in Python 3.1rc1 built from source on MacOS X 10.4.11 Tiger (Intel).

% ./python.exe Lib/test/test_asynchat.py test_close_when_done (main.TestAsynchat) ... ok test_empty_line (main.TestAsynchat) ... ok test_line_terminator1 (main.TestAsynchat) ... ok test_line_terminator2 (main.TestAsynchat) ... ok test_line_terminator3 (main.TestAsynchat) ... ok test_none_terminator (main.TestAsynchat) ... ok test_numeric_terminator1 (main.TestAsynchat) ... ok test_numeric_terminator2 (main.TestAsynchat) ... ok test_simple_producer (main.TestAsynchat) ... ok test_string_producer (main.TestAsynchat) ... ok test_close_when_done (main.TestAsynchat_WithPoll) ... ok test_empty_line (main.TestAsynchat_WithPoll) ... ok test_line_terminator1 (main.TestAsynchat_WithPoll) ... ok test_line_terminator2 (main.TestAsynchat_WithPoll) ... ok test_line_terminator3 (main.TestAsynchat_WithPoll) ... ok test_none_terminator (main.TestAsynchat_WithPoll) ... ok test_numeric_terminator1 (main.TestAsynchat_WithPoll) ... ok test_numeric_terminator2 (main.TestAsynchat_WithPoll) ... ok test_simple_producer (main.TestAsynchat_WithPoll) ... ok test_string_producer (main.TestAsynchat_WithPoll) ... ok test_find_prefix_at_end (main.TestHelperFunctions) ... ok test_basic (main.TestFifo) ... ok test_given_list (main.TestFifo) ... ok

Ran 23 tests in 7.726s

OK

msg112809 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2010-08-04 13:43

Josiah's fix in r73182 applies cleanly to the release26-maint branch, and fixes (for me, at least) the current OS X failures in test_asynchat and test_smtplib. I've attached a patch with the backport of this fix, and a Misc/NEWS entry.

Barry, can this go in before 2.6.6 final?

msg112811 - (view)

Author: Barry A. Warsaw (barry) * (Python committer)

Date: 2010-08-04 14:30

Thanks Mark, go ahead and apply this, then close this issue.

msg112813 - (view)

Author: Mark Dickinson (mark.dickinson) * (Python committer)

Date: 2010-08-04 14:42

Applied in r83718.

History

Date

User

Action

Args

2022-04-11 14:56:48

admin

set

github: 50048

2010-08-04 14:43:18

mark.dickinson

set

resolution: accepted -> fixed
stage: needs patch -> resolved

2010-08-04 14:42:53

mark.dickinson

set

status: open -> closed

messages: +

2010-08-04 14:30:31

barry

set

resolution: accepted
messages: +

2010-08-04 13:43:08

mark.dickinson

set

status: closed -> open
files: + issue5798_release26.patch

assignee: barry
versions: - Python 3.0, Python 3.1, Python 2.7
nosy: + barry

messages: +
resolution: fixed -> (no value)

2009-06-05 04:19:00

MrJean1

set

messages: +

2009-06-04 18:01:50

josiahcarlson

set

messages: +

2009-06-04 17:19:18

r.david.murray

set

messages: +

2009-06-04 17:08:10

MrJean1

set

messages: +

2009-06-04 00:06:08

ned.deily

set

messages: +

2009-06-03 19:49:29

josiahcarlson

set

status: open -> closed
resolution: fixed
messages: +

2009-06-03 19:49:05

josiahcarlson

set

files: - asyncore_fix_mac_2.patch

2009-06-03 16:55:47

benjamin.peterson

set

priority: high -> release blocker

2009-06-03 15:03:09

josiahcarlson

set

messages: +

2009-06-03 07:41:37

ned.deily

set

nosy: + benjamin.peterson, ned.deily
messages: +

2009-05-08 22:20:04

giampaolo.rodola

set

messages: +

2009-05-08 20:57:49

josiahcarlson

set

messages: +

2009-05-08 20:06:33

mark.dickinson

set

messages: +

2009-05-08 19:39:31

josiahcarlson

set

files: - unnamed

2009-05-08 19:38:56

josiahcarlson

set

files: - asyncore_fix_mac.patch

2009-05-08 19:38:45

josiahcarlson

set

files: + asyncore_fix_mac_2.patch

messages: +

2009-05-08 18:53:36

mark.dickinson

set

messages: +

2009-05-08 18:24:08

mark.dickinson

set

messages: +

2009-05-08 17:49:31

josiahcarlson

set

messages: +

2009-05-08 16🔞46

MrJean1

set

messages: +

2009-05-08 15:15:00

giampaolo.rodola

set

messages: +

2009-05-08 00:40:31

MrJean1

set

files: + unnamed

messages: +

2009-05-08 00🔞44

josiahcarlson

set

messages: +

2009-05-08 00:11:36

MrJean1

set

messages: +

2009-05-07 21:20:05

josiahcarlson

set

files: + asyncore_fix_mac.patch
keywords: + patch
messages: +

2009-05-07 20:30:15

mark.dickinson

set

messages: +

2009-05-07 20🔞05

josiahcarlson

set

messages: +

2009-05-07 20:15:02

josiahcarlson

set

messages: +

2009-05-07 20:10:52

mark.dickinson

set

messages: +

2009-05-07 20:03:02

josiahcarlson

set

messages: +

2009-05-07 19:52:44

josiahcarlson

set

messages: +

2009-05-07 19:28:06

mark.dickinson

set

messages: +

2009-05-07 19:10:09

mark.dickinson

set

messages: +

2009-05-07 18:30:05

josiahcarlson

set

messages: +

2009-05-07 17:27:43

r.david.murray

set

nosy: + r.david.murray, josiahcarlson, giampaolo.rodola
messages: +

type: behavior
stage: needs patch

2009-05-07 17:01:33

MrJean1

set

nosy: + MrJean1
messages: +

2009-05-07 16:50:28

mark.dickinson

set

nosy: + facundobatista, alanmcintyre
messages: +

2009-05-07 12:09:44

mark.dickinson

set

messages: +

2009-05-07 11:42:45

donmez

set

messages: +

2009-04-25 16:25:34

mark.dickinson

set

priority: high
versions: + Python 3.0, Python 3.1, Python 2.7
nosy: + mark.dickinson

messages: +

2009-04-20 11:02:11

donmez

create