[Python-Dev] New syntax for 'dynamic' attribute access (original) (raw)

Ben North ben at redfrontdoor.org
Mon Feb 12 00:45:05 CET 2007

Hi,

A few days ago I posted to python-ideas with a suggestion for some new Python syntax, which would allow easier access to object attributes where the attribute name is known only at run-time. For example:

setattr(self, method_name, getattr(self.metadata, method_name))

from Lib/distutils/dist.py could be rewritten

self.(method_name) = self.metadata.(method_name)

The new syntax would also be usable in augmented assignments, as in

obj.(attr_name) += 1

There was some discussion on python-ideas, which I've linked to in the draft PEP below. In particular, Guido van Rossum wrote:

I've thought of the same syntax. I think you should submit this to the PEP editor and argue on Python-dev for its inclusion in Python 2.6 -- there's no benefit that I see of waiting until 3.0.

so here I am. Does anybody have any opinions/suggestions, particularly on the "open questions" referred to in the draft PEP? To summarise these open questions:

Ben.

Abstract

Dynamic attribute access is currently possible using the "getattr"
and "setattr" builtins.  The present PEP suggests a new syntax to
make such access easier, allowing the coder for example to write

    x.('foo_%d' % n) += 1

    z = y.('foo_%d' % n).('bar_%s' % s)

instead of

    attr_name = 'foo_%d' % n
    setattr(x, attr_name, getattr(x, attr_name) + 1)

    z = getattr(getattr(y, 'foo_%d' % n), 'bar_%s' % s)

Note

(I wrote this patch mostly to advance my own understanding of and
experiment with the python language, but I've written it up in the
style of a PEP in case it might be a useful idea.)

Rationale

Dictionary access and indexing both have a friendly invocation
syntax: instead of x.__getitem__(12) the coder can write x[12].
This also allows the use of subscripted elements in an augmented
assignment, as in "x[12] += 1".  The present proposal brings this
ease-of-use to dynamic attribute access too.

Attribute access is currently possible in two ways:

* When the attribute name is known at code-writing time, the
  ".NAME" trailer can be used, as in

      x.foo = 42
      y.bar += 100

* When the attribute name is computed dynamically at run-time, the
  "getattr" and "setattr" builtins must be used:

      x = getattr(y, 'foo_%d' % n)
      setattr(z, 'bar_%s' % s, 99)

  The "getattr" builtin also allows the coder to specify a default
  value to be returned in the event that the object does not have
  an attribute of the given name:

      x = getattr(y, 'foo_%d' % n, 0)

This PEP describes a new syntax for dynamic attribute access ---
"x.(expr)" --- with examples given in the Abstract above.  The new
syntax also allows the provision of a default value in the "get"
case, as in:

    x = y.('foo_%d' % n, None)

This 2-argument form of dynamic attribute access is not permitted as
the target of an (augmented or normal) assignment.  Also, this part
of the new syntax was not as well received [6] in initial
discussions on python-ideas, and I agree that it does not fit so
cleanly.  I'm happy to prepare a revised PEP/patch without the
2-argument form if the consensus is that this would be preferred.

Finally, the new syntax can be used with the "del" statement, as in

    del x.(attr_name)

Impact On Existing Code

The proposed new syntax is not currently valid, so no existing
well-formed programs have their meaning altered by this proposal.

Across all "*.py" files in the 2.5 distribution, there are around
600 uses of "getattr", "setattr" or "delattr".  They break down as
follows (figures have some room for error because they were
arrived at by partially-manual inspection):

    c.300 uses of plain "getattr(x, attr_name)", which could be
          replaced with the new syntax;

    c.150 uses of the 3-argument form, i.e., with the default
          value; these could be replaced with the 2-argument form
          of the new syntax (the cases break down into c.125 cases
          where the attribute name is a literal string, and c.25
          where it's only known at run-time);

    c.5   uses of the 2-argument form with a literal string
          attribute name, which I think could be replaced with the
          standard "x.attribute" syntax;

    c.120 uses of setattr, of which 15 use getattr to find the
          new value; all could be replaced with the new syntax,
          the 15 where getattr is also involved would show a
          particular increase in clarity;

    c.5   uses which would have to stay as "getattr" because they
          are calls of a variable named "getattr" whose default
          value is the builtin "getattr";

    c.5   uses of the 2-argument form, inside a try/except block
          which catches AttributeError and uses a default value
          instead; these could use 2-argument form of the new
          syntax;

    c.10  uses of "delattr", which could use the new syntax.

As examples, the line

    setattr(self, attr, change_root(self.root, getattr(self, attr)))

from Lib/distutils/command/install.py could be rewritten

    self.(attr) = change_root(self.root, self.(attr))

and the line

    setattr(self, method_name, getattr(self.metadata, method_name))

from Lib/distutils/dist.py could be rewritten

    self.(method_name) = self.metadata.(method_name)

Performance Impact

Initial pystone measurements are inconclusive, but suggest there may
be a performance penalty of around 1% in the pystones score with the
patched version.  One suggestion is that this is because the longer
main loop in ceval.c hurts the cache behaviour, but this has not
been confirmed.  (Maybe a tool like valgrind [2] could help here?)

On the other hand, measurements suggest a speed-up of around 40--45%
for dynamic attribute access.

Discussion To Date

Initial posting of this PEP in draft form was to python-ideas on
20070209 [4], and the response was generally positive:

    I've thought of the same syntax. I think you should submit this
    to the PEP editor and argue on Python-dev for its inclusion in
    Python 2.6 -- there's no benefit that I see of waiting until
    3.0.  --- Guido van Rossum [5]

    Wow!  I have to say this is a compelling idea.  The syntax is a
    bit foreign looking, but [...] I feel like I could learn to like
    it anyway. --- Greg Falcon [6]

    I look forward to seeing this in Python 2.6.  --- Josiah
    Carlson, further down the thread [8]

with Greg Falcon expressing the reservations about the 2-argument
form already noted above, and Josiah Carlson raising a query about
performance:

    My only concern with your propsed change is your draft
    implementation. [...]  Specifically, your changes [...] may
    negatively affect general Python performance.  --- Josiah
    Carlson [7]

Some initial measurements (see above) suggest the performance
penalty is small, and Josiah Carlson said of such cost that it
"isn't really substantial". [8]

Questions To Be Resolved

* Whether to allow the 2-argument form for default arguments.

* Whether the performance impact is real; whether it is acceptable;
  whether alternative implementations might improve this aspect.

Alternative Syntax For The New Feature

Other syntaxes could be used, for example braces are currently
invalid in a "trailer", so could be used here, giving

    x{'foo_%d' % n} += 1

My personal preference is for the

    x.('foo_%d' % n) += 1

syntax though: the presence of the dot shows there is attribute
access going on; the parentheses have an analogous meaning to the
mathematical "work this out first" meaning.  This is also the
syntax used in the language Matlab [1] for dynamic "field" access
(where "field" is the Matlab term analogous to Python's
"attribute").

Discussions on python-ideas (see above) made no comment on the brace
alternative, and the .() notation was well-enough received, so the
brace alternative should be considered rejected, I think.

Error Cases

Only strings are permitted as attribute names, so for instance the
following error is produced:

  >>> x.(99) = 8
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: attribute name must be string, not 'int'

This is handled by the existing PyObject_GetAttr function.

Draft Implementation

A draft implementation adds a new alternative to the "trailer"
clause in Grammar/Grammar; a new AST type, "DynamicAttribute" in
Python.asdl, with accompanying changes to symtable.c, ast.c, and
compile.c, and three new opcodes (load/store/del) with
accompanying changes to opcode.h and ceval.c.  The patch consists
of c.180 additional lines in the core code, and c.100 additional
lines of tests.  It is available as sourceforge patch #1657573 [3].

References

[1] Using Dynamic Field Names :: Data Types (MATLAB Programming)

http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_prog/f2-41859.html

[2] Valgrind: "suite of tools for debugging and profiling Linux

programs" http://www.valgrind.org/

[3] Sourceforge patch #1657573

http://sourceforge.net/tracker/index.php?func=detail&aid=1657573&group_id=5470&atid=305470

[4]

http://mail.python.org/pipermail/python-ideas/2007-February/000210.html

[5]

http://mail.python.org/pipermail/python-ideas/2007-February/000211.html

[6]

http://mail.python.org/pipermail/python-ideas/2007-February/000212.html

[7]

http://mail.python.org/pipermail/python-ideas/2007-February/000213.html

[8]

http://mail.python.org/pipermail/python-ideas/2007-February/000227.html

Copyright

This document has been placed in the public domain.

More information about the Python-Dev mailing list