[Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered (original) (raw)

MRAB python at mrabarnett.plus.com
Tue Sep 13 13:59:37 EDT 2016

On 2016-09-13 11:44, Nick Coghlan wrote:

On 13 September 2016 at 08:25, Gregory P. Smith <greg at krypto.org> wrote:

At this point I think coding up an example patch against beta1 offering a choice of disordered iteration capability that does not increase memory or iteration overhead in any significant way is needed.

The problem is... I don't know how to express this as an API. Which sinks my whole though process and tables the idea. A parameter to .items(), .keys() and .values() is undesirable as it isn't backwards compatible [meaning it'll never be used] and .keys() needs to match iter which can't have one anyways. A parameter on dict construction is similarly infeasible. Requiring the use of an orderdict like type in order to get the behavior is undesirable. Effectively I'm asking for some boolean state in each dict as to if it should iterate in order or not and a way to expose that to pure Python code in a way that namespace dicts iterate in order by default and others do not unless explicitly configured to do so. oh well. end thought process on my end. it was good while it lasted. I think this is looking at the compatibility testing problem from the wrong direction anyway, as rather than making it difficult for people to implicitly depend on the default key ordering, the scenario we would want to help with is this one: 1. Library developer inadvertently depends on the dicts-are-ordered-by-default implementation detail 2. Library user reports "your library isn't working for me on " 3. Library developer figures out the problem, and would like to update their test suite to deliberately provoke the misbehaviour 4. ??? That is, it falls into the same category as folks depending on CPython's reference counting for prompt resource cleanup, where we offer ResourceWarning to detect such cases, and context managers to clean them up more explicitly. For dict ordering dependence, anyone regularly testing against CPython 2.7 and CPython 3.5 will already have a good chance of detecting key order reliance just through hash randomisation (e.g. I hit an "inconsistent key order in generated JSON makes line-based diffing unreadable" one myself last week with a 3-entry dict for md5, sha1 and sha256 hashes - it was relatively rare to get the same key order two runs in a row) That means the only problematic case is the one where the only CPython version a project supports is 3.6+ and they want to support alternate implementations that don't preserve order in their default dict implementation. Given that current alternate implementations are still in the process of catching up to 3.5 (or Python 3 at all in the case of Jython and IronPython), I think we still have a good few years to ponder the question before this particular concern starts cropping up in practise :) The recommended way of dealing with features across different versions of Python is to check for them and see if they raise NameError or whatever, but I wonder if there would be any benefit to recording such things somewhere, e.g. sys.features['ordered_args'] returns True if arguments are passed in an ordered dict.

More information about the Python-Dev mailing list