Removing the threading.Lock locks and replacing them with RLock objects to avoid deadlocks. by petyaslavova · Pull Request #3677 · redis/redis-py (original) (raw)

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

Pull Request check-list

Please make sure to review and check all of these items:

Do tests and lints pass with this change?
Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
Is the new or changed code fully tested?
Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
Is there an example added to the examples folder (if applicable)?

NOTE: these things are not required to open a PR and can be done afterwards / while the PR is open.

Description of change

Usage of threading.Lock has introduced deadlocks in the client.
With this change, I'm removing all places that can cause such issues.
The change is breaking, because it changes the interfaces of several objects and cannot be released before the next major release, which will be 7.0.0

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR replaces all usages of threading.Lock with threading.RLock to prevent deadlocks, updating both tests and core locking interfaces. It’s a breaking change that adjusts constructor signatures and type hints to expect reentrant locks.

Swapped Lock for RLock in multiple test files to align with new behavior.
Updated constructor parameters and type annotations in redis/event.py, redis/connection.py, and redis/client.py to use RLock.
Removed legacy non-reentrant lock branches and related TODOs in redis/connection.py.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/test_credentials.py	Changed mock pools’ _lock initialization from Lock to RLock.
tests/test_connection.py	Updated CacheProxyConnection instantiation to pass an RLock.
tests/test_cluster_transaction.py	Replaced mock_pool._lock = Lock() with RLock().
redis/event.py	Refactored connection_lock parameter/type to RLock.
redis/connection.py	Adjusted pool_lock type to RLock; removed non-reentrant fallback code.
redis/client.py	Swapped client locks to RLock and removed outdated TODOs.

Comments suppressed due to low confidence (3)

tests/test_credentials.py:326

[nitpick] Add a unit test that acquires the same RLock twice in a row to verify reentrant behavior and ensure no deadlock occurs under nested acquisitions.

mock_pool._lock = threading.RLock()

redis/connection.py:813

Update the constructor docstring for this class to document the change from Lock to RLock in the pool_lock parameter, since this is now part of the public API and a breaking change.

pool_lock: threading.RLock,

tests/test_connection.py:496

[nitpick] Consider extracting the repeated instantiation of CacheProxyConnection with an RLock into a pytest fixture to reduce duplication and improve clarity across tests.

proxy_connection = CacheProxyConnection(mock_connection, mock_cache, threading.RLock())

This PR fixes issue #3640

@petyaslavova - appreciate this is a breaking change but given the deadlocks introduced has broken production code for us (and, based on feedback on various OS repos, many others), do you have an estimate of when a v7.0.0 will be released?

To avoid the deadlocks we had to downgrade from 6.x to 5.2.1 and I guess everyone on >5.2.1,<7.0.0 may potentially also have very hard to debug issues going forward (it took us forever to triangulate the issue to deadlocks in redis)?

ManelCoutinhoSensei pushed a commit to ManelCoutinhoSensei/redis-py that referenced this pull request

Jun 30, 2025

ManelCoutinhoSensei pushed a commit to ManelCoutinhoSensei/redis-py that referenced this pull request

Jul 1, 2025

This was referenced

Aug 17, 2025