[Python-Dev] PEP 487: Simpler customization of class creation (original) (raw)

Nikita Nemkin nikita at nemkin.ru
Mon Jun 20 14:31:54 EDT 2016

On Mon, Jun 20, 2016 at 9:48 PM, Guido van Rossum <guido at python.org> wrote:

On Thu, Jun 16, 2016 at 3:24 PM, Nikita Nemkin <nikita at nemkin.ru> wrote:

I didin't know that PyPy has actually implemented packed ordered dicts! https://morepypy.blogspot.ru/2015/01/faster-more-memory-efficient-and-more.html https://mail.python.org/pipermail/python-dev/2012-December/123028.html This old idea by Raymond Hettinger is vastly superior to definitionorder duct tape (now that PyPy has validated it). It also gives kwarg order for free, which is important in many metaprogramming scenarios. Not to mention memory usage reduction and dict operations speedup... That idea is only vastly superior if we want to force all other Python implementations to also have an order-preserving dict with the same semantics and API.

Right. Ordered by default is a very serious implementation constraint. It's only superior in a sense that it completely subsumes/obsoletes PEP 520.

I'd like to hear more about your metaprogramming scenarios -- often such things end up being code the author is ashamed of. Perhaps they should stay in the shadows? Or could we do something to make it so you won't have to be ashamed of it?

What I meant is embedding declarative domain-specific languages in Python. Examples of such languages include SQL table definitions, binary data definitions (in-memory C structs or wire protocol), GUI definitions (look up enaml for an interesting example), etc. etc. DSLs are a well defined field and the point of embedding into Python is to implement in Python and to empower DSL with Python constructs for generation and logic.

Basic blocks for a declarative language are lists and "objects" - groups of ordered, named fields.

Representing lists is easy and elegant, commas make a tuple and [] makes a list.

It's when trying to represent "objects" the issues arise. Literal dicts are "ugly" (for DSL purposes) and unordered. Lists of 2-tuples are even uglier. Py3 gave us prepare for ordered class bodies, and this became a first valid option. For example, SQL table:

class MyTable(SqlTable):
    field1 = Type1(options...)
    field2 = Type2()

Unfortunately, class declarations don't look good when nested, and nesting is a common thing.

class MainWindow:
    caption = "Window"
    class HSplit:
        label1 = Label(...)
        text1 = Text(...)

You get the idea. Another option for expressing "objects" are function calls with kwargs:

packet = Struct(type=uint8,
                length=uint32,
                body=Array(uint8, 'type'))

Looks reasonably clean, but more often than not requires kwargs to be ordered. THIS is the scenario I was talking about.

Function attributes also have a role, but being attached to function definitions, their scope is somewhat limited.

Of course, all of the above is largely theoretical, for two basic reasons:

  1. Python syntax/runtime is too rigid for a declarative DSL. (Specifically, embedded DSL. The syntax alone can be re-used with ast.parse, but it's a different scenario.)
  2. DSLs in general are grossly unpythonic, hiding loads of magic and unfamiliar semantics behind what looks like a normal Python. It's not something to be ashamed of, but the benefit rarely justifies the (maintenance) cost.

To be clear: I'm NOT advocating for ordered kwargs. Embedding DSLs into Python is generally a bad idea.

PS. prepare enables many DSL tricks. In fact, it's difficult to imagine a use case that's not related to some attempt at DSL. Keyword-only args also help: ordered part of the definition can go into *args, while attributes/options are kw-only args.

More information about the Python-Dev mailing list