[Python-Dev] What do we do about bad slicing and possible crashes (issue 27867) (original) (raw)

Serhiy Storchaka storchaka at gmail.com
Tue Aug 30 10:41:29 EDT 2016

On 30.08.16 15:31, Dima Tisnek wrote:

On 30 August 2016 at 14:13, Serhiy Storchaka <storchaka at gmail.com> wrote:

1. Detect length change and raise.

It would be simpler solution. But I afraid that this can break third-party code that "just works" now. For example slicing a list "just works" if step is 1. It can return not what the author expected if a list grows, but it never crashes, and existing code can depends on current behavior. This solution is not applicable in maintained versions. Serhiy, If dictionary is iterated in thread1 while thread2 changes the dictionary, thread1 currently raises RuntimeError. Would cloning current dict behaviour to slice with overridden index make sense?

No, these are different things. The problem with dict iterating is unavoidable, but slicing can be defined consistently (as described by Terry in option 2). Changing a dict can change the order and invalidates iterators (except trivial cases of just created or finished iterators). But slicing can be atomic (and it is atomic de facto in many cases), we just should call all index-es before looking on a sequence.

I'd argue 3rd party code depends on slicing not to raise an exception, is same as 3rd party code depending on dict iteration not to raise and exception; If same container may be concurrently used in another thread, then 3rd party code is actually buggy. It's OK to break such code.

We shouldn't break third-party code in maintained releases. De facto slicing is atomic now in many cases, and it is nowhere documented that it is not atomic. The problem only with using broken by design PySlice_GetIndicesEx() in CPython. If CPython would implemented without PySlice_GetIndicesEx() (with more cumbersome code), it is likely this issue wouldn't be raised.

More information about the Python-Dev mailing list