| msg189302 - (view) |
Author: Andre Dias (aod) |
Date: 2013-05-15 20:41 |
| The problem is that Queue.empty() is True even if Queue.qsize()>0! #!/usr/bin/python from multiprocessing import Queue numbers=Queue() for i in range (0,10): numbers.put(i) if numbers.qsize()>0 and numbers.empty(): print "BUG?!" |
|
|
| msg189305 - (view) |
Author: Richard Oudkerk (sbt) *  |
Date: 2013-05-15 21:19 |
| From the docs: qsize() Return the approximate size of the queue. Because of multithreading/multiprocessing semantics, this number is not reliable. Adding a short sleep before calling qsize() and empty() should make things appear to work. But really, there are no good reasons for using qsize() except for debugging. The same applies to queue.Queue. |
|
|
| msg189309 - (view) |
Author: Andre Dias (aod) |
Date: 2013-05-15 21:25 |
| But qsize() is working. what is not working is empty() 2013/5/15 Richard Oudkerk <report@bugs.python.org> > > Richard Oudkerk added the comment: > > >From the docs: > > qsize() > Return the approximate size of the queue. Because of > multithreading/multiprocessing semantics, this number > is not reliable. > > Adding a short sleep before calling qsize() and empty() should make things > appear to work. > > But really, there are no good reasons for using qsize() except for > debugging. The same applies to queue.Queue. > > ---------- > nosy: +sbt > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17985> > _______________________________________ > |
|
|
| msg189315 - (view) |
Author: Richard Oudkerk (sbt) * |
Date: 2013-05-15 22:24 |
| On 15/05/2013 10:25pm, Andre Dias wrote: > But qsize() is working. what is not working is empty() empty() returns False when there is data in the underlying pipe. But the data does not enter the pipe until a background thread has written it to the pipe. This should not cause any problems. Using Queue.empty() is always subject to races. It is just that the multiprocessing version has an additional type of race compared to the normal one. |
|
|
| msg189318 - (view) |
Author: Andre Dias (aod) |
Date: 2013-05-15 22:33 |
| RIchard, But the example program has no races, no threads, nothing. empty() is returning TRUE even though qsize() is >0 (which actually is) And it happens almost every time I run that small example. I had read the module doc, and I know its an unreliable method, but man, the example program is too simple to fail Truth is I decided qsize() is more reliable and im using it in my programs, but man empty() problem is so ridiculous that I decided to submit here 2013/5/15 Richard Oudkerk <report@bugs.python.org> > > Richard Oudkerk added the comment: > > On 15/05/2013 10:25pm, Andre Dias wrote: > > But qsize() is working. what is not working is empty() > > empty() returns False when there is data in the underlying pipe. But > the data does not enter the pipe until a background thread has written > it to the pipe. This should not cause any problems. > > Using Queue.empty() is always subject to races. It is just that the > multiprocessing version has an additional type of race compared to the > normal one. > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17985> > _______________________________________ > |
|
|
| msg189322 - (view) |
Author: Richard Oudkerk (sbt) * |
Date: 2013-05-15 22:46 |
| On 15/05/2013 11:33pm, Andre Dias wrote: > But the example program has no races, no threads, nothing. > empty() is returning TRUE even though qsize() is >0 (which actually is) > And it happens almost every time I run that small example. > I had read the module doc, and I know its an unreliable method, but man, > the example program is too simple to fail > Truth is I decided qsize() is more reliable and im using it in my programs, > but man empty() problem is so ridiculous that I decided to submit here If you assume everything is single-threaded then you would assume that this is too simple to fail. But there *is* more than thread involved. empty() and qsize() are basically redundant. You would almost certainly be better off using get_noblock()/put_noblock() and catching Empty/Full. |
|
|
| msg191794 - (view) |
Author: Richard Oudkerk (sbt) * |
Date: 2013-06-24 18:25 |
| This is really a documentation issue. The doc fix for #18277 covers this. |
|
|