Issue 15330: allow deque to act as a thread-safe circular buffer (original) (raw)

This issue has been migrated to GitHub: https://github.com/python/cpython/issues/59535

classification

process

Created on 2012-07-11 22:20 by chris.jerdonek, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (6)
msg165276 - (view)	Author: Chris Jerdonek (chris.jerdonek) *	Date: 2012-07-11 22:20
It seems like it would be useful if collections.deque had a thread-safe method that could rotate(1) and return the rotated value. This would let deque to act as a thread-safe circular buffer (e.g. if serving jobs to multiple threads in a loop, like `python -m test --forever`).
msg165279 - (view)	Author: Raymond Hettinger (rhettinger) *	Date: 2012-07-12 05:27
By thread-safe you mean that the operation should be atomic?
msg165280 - (view)	Author: Chris Jerdonek (chris.jerdonek) *	Date: 2012-07-12 05:37
Yes, atomic. I was under the impression that the existing deque.rotate() is atomic, in which case deque.rotate(1) almost provides what I'm suggesting (lacking only the return value).
msg165281 - (view)	Author: Raymond Hettinger (rhettinger) *	Date: 2012-07-12 05:47
We make almost no guarantees about atomicity. If something in CPython happens to be atomic, it is not guaranteed to hold in other implementations. The recommendation is to use locks anywhere you need a critical section of code. FWIW, the term thread-safe is taken to mean, "won't break in a multi-threaded environment". That is a bit different from atomic. For example, the decimal module was designed with thread local contexts, but there are no atomic decimal operations.
msg165282 - (view)	Author: Chris Jerdonek (chris.jerdonek) *	Date: 2012-07-12 06:04
Thanks for the info. A couple questions: what does "won't break" mean -- that it won't throw an exception of a type that it wouldn't normally throw in a single-threaded environment? And does this mean that not even deque.pop() is atomic?
msg165315 - (view)	Author: Raymond Hettinger (rhettinger) *	Date: 2012-07-12 15:58
> A couple questions: what does "won't break" mean That means that its internal invariants always survive in a multi-threaded environment. > that it won't throw an exception of a type that it > wouldn't normally throw in a single-threaded environment? Unless that exception is being raised to indicate that a resource is in use (much that same as a database notifying you that it can't make a change because the DB is currently locked).