[Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Tue Jul 18 12:16:30 EDT 2017

On Tue, 18 Jul 2017 09:08:08 -0700 Ethan Furman <ethan at stoneleaf.us> wrote:

Nick Coughlan: -------------

It is "Nick Coghlan" not "Coughlan".

> As another example of this: while trading the global import lock for > per-module locks eliminated most of the old import deadlocks, it turns > out that it also left us with some fairly messy race conditions and > more fragile code (I still count that particular case as a win > overall, but it definitely raises the barrier to entry for maintaining > that code). > > Unfortunately, these are frequently cases where the benefits are > immediately visible (e.g. faster benchmark results, removing > longstanding limitations on user code), but the downsides can > literally take years to make themselves felt (e.g. higher defect rates > in the interpreter, subtle bugs in previously correct user code that > are eventually traced back to interpreter changes).

I'll reply here again: the original motivation for the per-module import lock was not performance but correctness.

The import deadlocks were really in the category of "subtle bugs" that only occur in certain timing conditions (especially when combined with PyImport_ImportModuleNoBlock and/or stdlib modules which can try to import stuff silently, such as the codecs module). So we traded a category of "subtle bugs" due to a core design deficiency for another category of "subtle bugs" due to an imperfect implementation, the latter being actually fixable incrementally :-)

Disclaimer: I wrote the initial per-module lock implementation.

Regards

Antoine.

More information about the Python-Dev mailing list