[Python-Dev] PEP520 and absence of definition_order (original) (raw)

Guido van Rossum guido at python.org
Sat Sep 10 17:26:58 EDT 2016

On Sat, Sep 10, 2016 at 10:57 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

On 11 September 2016 at 03:08, Guido van Rossum <guido at python.org> wrote:

So I'm happy to continue thinking about this, but I expect this is not such a big deal as you fear. Anyway, let's see if someone comes up with a more convincing argument by beta 2! For CPython specifically, I don't have anything more convincing than Ethan's Enum example (where the way the metaclass works means most of the interesting attributes don't live directly in the class dict, they live in private data structures stored in the class dict, making "list(MyEnum.dict)" inherently uninteresting, regardless of whether it's ordered or not).

But that would only matter if we also defined a helper utility that used definition_order. I expect that the implementation of Enum could be simplified somewhat in Python 3.6 since it can trust that the namespace passed into new is ordered (so it doesn't have to switch it to an OrderedDict in prepare, perhaps).

In any case the most likely way to use definition_order in general was always to filter its contents through some other condition (e.g. "isn't a method and doesn't start with underscore") -- you can do the same with keys(). Classes that want to provide a custom list of "interesting" attributes can provide that using whatever class method or attribute they want -- it's just easier to keep those attributes ordered because the namespace is always ordered.

The proxy use cases I'm aware of (wrapt, weakref.proxy) tend to be used to wrap normal instances rather than class objects themselves, so they shouldn't be affected.

With ordered-by-default class namespaces, both heap types and non-heap types should also mostly be populated in the "logical order" (i.e. the order names appear in the relevant C arrays), but that would formally be an implementation detail at this point, rather than something we commit to providing. The only other argument that occurs to me is one that didn't come up in the earlier PEP 520 discussions: how a not-quite-Python implementation (or a Python 3.5 compatible implementation that doesn't offer order-preserving behaviour the way PyPy does) can make sure that code that relies on ordered class namespaces fails in an informative way when run on that implementation.

Is that a real use case? It sounds like you're just constructing an artificial example that would be less convenient without definition_order.

With definitionorder, that's straightforward - the code that needs it will fail with AttributeError, and searching for the attribute named in the exception will lead affected users directly to PEP 520 and the Python 3.6 What's New guide.

But that code would have to be written to use definition_order. It could just as easily be written to assert that sys.version_info() >= (3, 6).

With implicitly ordered class namespaces, you don't get an exception if the namespace isn't actually order preserving - you get attributes in an arbitrary order instead. Interpreters can't detect that the user specifically wanted order preserving behaviour, and library and application authors can't readily detect whether or not the runtime offers order preserving behaviour (since they may just get lucky on that particular run).

That sounds very philosophical. You still can't check whether dict is order-preserving -- all you can do is checking whether a class preserves its order. Since PEP 520 is accepted only for Python 3.6, checking for the presence of definition_order is no different than checking the version.

Even if we added a new flag to sys.implementation that indicated the use of order preserving class namespaces, there'd still be plenty of scope for subtle bugs where libraries and frameworks weren't checking that flag before relying on the new behaviour.

OK, I'm beginning to see the argument here. You want all code that relies on the order to be explicitly declaring that it does so by using a new API.

Unfortunately the mere presence of definition_order doesn't really help here -- since all dicts are order-preserving, there's still nothing (apart from documentation) to stop apps from relying on the ordering of the class dict directly.

Cheers, Nick.

P.S. I'd actually love it if we could skip definitionorder - there really is a whole lot of runtime clutter on class objects, and we're adding annotations as well. Unfortunately, I also think we made the right call the first time around in thinking it would still be necessary even if class namespaces became order preserving :)

Note that annotations is only added when there are annotations, so its presence could be used as a flag of sorts. (However you shouldn't use it directly -- each class in the MRO has its own annotations, and you should use typing.get_type_hints(cls) to coalesce all of them.)

-- --Guido van Rossum (python.org/~guido)

More information about the Python-Dev mailing list