[Python-Dev] Guarantee ordered dict literals in v3.7? (original) (raw)

Chris Barker - NOAA Federal chris.barker at noaa.gov
Tue Nov 7 19:15:53 EST 2017

This seems like overkill to me. By the same logic, we should add a

"delay garbage collection" mode, that allows people to test that their

code doesn't make unwarranted assumptions that a reference-counting

garbage collector is in use.

Actually, there is a LOT of code out there that expects reference counting. I know a lot of my code does. So if cPython does abandon it some day, there will be issues.

Another thought:

Let’s say Python declares that dict literals are order preserving.

Then, one day, someone invents a massively more efficient non-order preserving implementation, and we want to use it for Python 4.

So the Python 4 language spec says that dicts are not order preserving. And this is flagged as an INCOMPATIBLE CHANGE.

Now everyone knows to go and check their code, and the 3to4 tool adds an “o” to all dict literals.

People will complain, but it won’t be unexpected breakage.

Compare to leaving it as an implementation detail — now we will have a lot of code in the wild that expects order-preservation (because people don’t read the language spec) that will break with such a change without any real warning.

I think we really do need to accept that cPython is a reference implementation.

Because it is.

By the way, I only just realized I can delete a key to demonstrate non-order-preservation on py 3.6. So at least I know what to tell students now.

-CHB

But you can get pretty fine-grained control of garbage collection with judicious use of gc.disable(). If there were a similar mechanism for changing the ordering properties of dictionaries in code, I wouldn't suggest it as an interpreter flag / option.

And you're right - it's not pressing - the people likely to test with dictionaries scrambled are exactly the people likely to be supporting 2.7 and 3.5, but that won't be true forever, and it would be nice to have some mechanism to test that you're not relying on this property.

@Barry Warsaw

As has been suggested elsewhere, if we decide not to make that guarantee, then we should probably deliberately randomize iteration order.

This was my suggestion of a middle way, since deliberate randomization seems like a step too far just to avoid people relying on implementation details. I've seen plenty of code that assumes that assert statements will always throw AssertionError, but that's not guaranteed, and some people run their test suites with -O just to check that they aren't making that assumption.

On 11/07/2017 04:15 PM, Paul Moore wrote:

On 7 November 2017 at 20:35, Paul G <paul at ganssle.io> wrote:

If dictionary order is not guaranteed in the spec and the dictionary order isn't randomized (which I think everyone agrees is a bit messed up), it would probably be useful if you could enable "random order mode" in CPython, so you can stress-test that your code isn't making any assumptions about dictionary ordering without having to use an implementation where order isn't deterministic.

I could either be something like an environment variable SCRAMBLE_DICT_ORDER or a flag like --scramble-dict-order. That would probably help somewhat with the very real problem of "everyone's going to start counting on this ordered property".

Most public projects (which are the only ones that really need to

worry about this sort of detail) will probably be supporting Python

3.5 and likely even Python 2.7 for some time yet. So they test under

non-order-preserving dictionary implementations anyway. And if code

that's only targeted for Python 3.7 assumes order preserving

dictionaries, it's likely not got a huge user base anyway, so what's

the problem?

Paul

Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20171107/369e2c60/attachment.html>

More information about the Python-Dev mailing list