Issue 2808: asynchat forgets packets when push is called from a thread (original) (raw)

Issue2808

Created on 2008-05-10 11:00 by xix xeaon, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (3)
msg66510 - (view)	Author: xix xeaon (xix xeaon)	Date: 2008-05-10 10:59
okay, my first bug post, hope I do it right. if you push more than 512 bytes () from a thread then only the first packet will be sent, all other packets will be forgotten. the remaining packets will be sent when something else happens, like a new connection is opened, or more input data is read. it's also sent when the select times out (default 30 seconds) and starts over. if push is called twice then the first two packets will be sent but the rest will be forgotten. if this problem arises in your code then you can easily fix it with a 0 second timeout for loop(), or preferably something like 0.0001 so that the cpu stays calm ;) you should be able to reproduce this bug with the attached file, just run it and then connect with netcat like: echo "\n" | nc localhost 1100 netcat will print almost 6 and a half lines of x and then the rest after 5 seconds (30 seconds is boring..) or if you have wireshark up then you'll see it easily. and would someone please apply all the patches for this module! =)
msg67167 - (view)	Author: Josiah Carlson (josiahcarlson) *	Date: 2008-05-21 19:00
My suggestion: don't do that. Asynchronous sockets, and asyncore/related libraries are not designed for, nor intended to be used as part of a threaded IO application. Why? Because most protocols are very concerned with data ordering, and sending from multiple threads can cause *serious* issues. I do not believe that this should change. Note that you can work around this limitation by using something like the following, but again, this is not suggested (you should instead work asyncore.poll() calls into some sort of main loop within your application). from Queue import Queue check_threads = 0 class my_async(asyncore.dispatcher): def __init__(self, *args, **kwargs): self.q = Queue() asyncore.dispatcher.__init__(self, *args, **kwargs) def push_t(self, data): global check_threads self.q.put(data) check_threads = 1 def handle_threaded_push(self): while not self.q.empty(): self.push(self.q.get()) def loop(timeout=.1, map=None): global check_threads if map is None: map = asyncore.socket_map while 1: asyncore.poll(timeout, map) if check_threads: check_threads = 0 for v in map.values(): try: v.handle_threaded_push() except: #handle exceptions better here pass
msg67175 - (view)	Author: xix xeaon (xix xeaon)	Date: 2008-05-21 22:03
I have good reason to use this combination of asynchat and threads (in a way which doesn't cause any of these serious issues you speak of), but if you don't want it fixed then that's fine with me since I know how to work around it, I just reported it =P