[Python-ideas] New syntax for 'dynamic' attribute access (original) (raw)

Guido van Rossum guido at python.org
Fri Feb 9 16:51:47 CET 2007

I've thought of the same syntax. I think you should submit this to the PEP editor and argue on Python-dev for its inclusion in Python 2.6 -- there's no benefit that I see of waiting until 3.0.

--Guido

On 2/9/07, Ben North <ben at redfrontdoor.org> wrote:

Hi,

I'd like to describe an addition I made to Python syntax which allows easier access to attributes where the attribute name is only known at run-time. For example: setattr(self, methodname, getattr(self.metadata, methodname)) from Lib/distutils/dist.py could be rewritten self.(methodname) = self.metadata.(methodname) As noted in the PEP-style description below, I mostly did this for fun, but I thought it might be worth bringing to the attention of python-ideas. A quick search through prior postings and Google for this idea didn't come up with anything. Ben.

- - - - 8< - - - -_ _PEP: XXX_ _Title: Syntax For Dynamic Attribute Access_ _Version: RevisionRevisionRevision Last-Modified: DateDateDate Author: Ben North <ben at redfrontdoor.org> Status: Draft Type: Standards Track Content-Type: text/plain Created: 29-Jan-2007 Post-History: Abstract Dynamic attribute access is currently possible using the "getattr" and "setattr" builtins. The present PEP suggests a new syntax to make such access easier, allowing the coder for example to write x.('foo%d' % n) += 1 z = y.('foo%d' % n).('bar%s' % s) instead of attrname = 'foo%d' % n setattr(x, attrname, getattr(x, attrname) + 1) z = getattr(getattr(y, 'foo%d' % n), 'bar%s' % s) Note (I wrote this patch mostly to advance my own understanding of and experiment with the python language, but I've written it up in the style of a PEP in case it might be a useful idea.) Rationale Dictionary access and indexing both have a friendly invocation syntax: instead of x.getitem(12) the coder can write x[12]. This also allows the use of subscripted elements in an augmented assignment, as in "x[12] += 1". The present proposal brings this ease-of-use to dynamic attribute access too. Attribute access is currently possible in two ways: * When the attribute name is known at code-writing time, the ".NAME" trailer can be used, as in x.foo = 42 y.bar += 100 * When the attribute name is computed dynamically at run-time, the "getattr" and "setattr" builtins must be used: x = getattr(y, 'foo%d' % n) setattr(z, 'bar%s' % s, 99) The "getattr" builtin also allows the coder to specify a default value to be returned in the event that the object does not have an attribute of the given name: x = getattr(y, 'foo%d' % n, 0) This PEP describes a new syntax for dynamic attribute access --- "x.(expr)" --- with examples given in the Abstract above. The new syntax also allows the provision of a default value in the "get" case, as in: x = y.('foo%d' % n, None) This 2-argument form of dynamic attribute access is not permitted as the target of an (augmented or normal) assignment. Finally, the new syntax can be used with the "del" statement, as in del x.(attrname) Impact On Existing Code The proposed new syntax is not currently valid, so no existing well-formed programs have their meaning altered by this proposal. Across all "*.py" files in the 2.5 distribution, there are around 600 uses of "getattr", "setattr" or "delattr". They break down as follows (figures have some room for error because they were arrived at by partially-manual inspection): c.300 uses of plain "getattr(x, attrname)", which could be replaced with the new syntax; c.150 uses of the 3-argument form, i.e., with the default value; these could be replaced with the 2-argument form of the new syntax (the cases break down into c.125 cases where the attribute name is a literal string, and c.25 where it's only known at run-time); c.5 uses of the 2-argument form with a literal string attribute name, which I think could be replaced with the standard "x.attribute" syntax; c.120 uses of setattr, of which 15 use getattr to find the new value; all could be replaced with the new syntax, the 15 where getattr is also involved would show a particular increase in clarity; c.5 uses which would have to stay as "getattr" because they are calls of a variable named "getattr" whose default value is the builtin "getattr"; c.5 uses of the 2-argument form, inside a try/except block which catches AttributeError and uses a default value instead; these could use 2-argument form of the new syntax; c.10 uses of "delattr", which could use the new syntax. As examples, the line setattr(self, attr, changeroot(self.root, getattr(self, attr))) from Lib/distutils/command/install.py could be rewritten self.(attr) = changeroot(self.root, self.(attr)) and the line setattr(self, methodname, getattr(self.metadata, methodname)) from Lib/distutils/dist.py could be rewritten self.(methodname) = self.metadata.(methodname) Alternative Syntax For The New Feature Other syntaxes could be used, for example braces are currently invalid in a "trailer", so could be used here, giving x{'foo%d' % n} += 1 My personal preference is for the x.('foo%d' % n) += 1 syntax though: the presence of the dot shows there is attribute access going on; the parentheses have an analogous meaning to the mathematical "work this out first" meaning. This is also the syntax used in the language Matlab [1] for dynamic "field" access (where "field" is the Matlab term analogous to Python's "attribute"). Error Cases Only strings are permitted as attribute names, so for instance the following error is produced: >>> x.(99) = 8 Traceback (most recent call last): File "", line 1, in TypeError: attribute name must be string, not 'int' This is handled by the existing PyObjectGetAttr function. Draft Implementation A draft implementation adds a new alternative to the "trailer" clause in Grammar/Grammar; a new AST type, "DynamicAttribute" in Python.asdl, with accompanying changes to symtable.c, ast.c, and compile.c, and three new opcodes (load/store/del) with accompanying changes to opcode.h and ceval.c. The patch consists of c.180 additional lines in the core code, and c.100 additional lines of tests. References [1] Using Dynamic Field Names :: Data Types (MATLAB Programming) http://www.mathworks.com/access/helpdesk/help/techdoc/matlabprog/f2-41859.html Copyright This document has been placed in the public domain. [PAGE-BREAK GOES HERE BUT REMOVED FOR EMAIL] Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: - - - - 8< - - - -_ _diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG_Python-2.5/Grammar/Grammar Python-2.5/Grammar/Grammar_ _--- ORIG_Python-2.5/Grammar/Grammar 2006-05-25 12:25:51.000000000 +0100_ _+++ Python-2.5/Grammar/Grammar 2007-02-01 18:07:04.133160000 +0000_ _@@ -119,7 +119,7 @@_ _listmaker: test ( listfor | (',' test)* [','] )_ _testlistgexp: test ( genfor | (',' test)* [','] )_ _lambdef: 'lambda' [varargslist] ':' test_ _-trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME_ _+trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME | '.' '(' test [',' test] ')'_ _subscriptlist: subscript (',' subscript)* [',']_ _subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop]_ _sliceop: ':' [test]_ _diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG_Python-2.5/Include/opcode.h Python-2.5/Include/opcode.h_ _--- ORIG_Python-2.5/Include/opcode.h 2006-02-27 22:32:47.000000000 +0000_ _+++ Python-2.5/Include/opcode.h 2007-02-02 13:36:58.542848000 +0000_ _@@ -39,6 +39,10 @@_ _#define SLICE 30_ _/* Also uses 31-33 */_ _+/* LOADDYNAMICATTR is below because it takes an argument. */_ _+#define STOREDYNAMICATTR 36_ _+#define DELETEDYNAMICATTR 37_ _+_ _#define STORESLICE 40_ _/* Also uses 41-43 */_ _@@ -89,6 +93,9 @@_ _#define UNPACKSEQUENCE 92 /* Number of sequence items */_ _#define FORITER 93_ _+#define LOADDYNAMICATTR 94 /* Whether default given; 0 = no, 1 = yes */_ _+/* STOREDYNAMICATTR, DELETEDYNAMICATTR are above; they take no argument. */_ _+_ _#define STOREATTR 95 /* Index in name list */_ _#define DELETEATTR 96 /* "" */_ _#define STOREGLOBAL 97 /* "" */_ _diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG_Python-2.5/Lib/test/testdynattr.py Python-2.5/Lib/test/testdynattr.py_ _--- ORIG_Python-2.5/Lib/test/testdynattr.py 1970-01-01 01:00:00.000000000 +0100_ _+++ Python-2.5/Lib/test/testdynattr.py 2007-02-02 17:37:37.982390000 +0000_ _@@ -0,0 +1,91 @@_ _+import unittest_ _+import warnings_ _+import sys_ _+from test import testsupport_ _+_ _+class AttrHolder:_ _+ pass_ _+_ _+class TestDynAttr(unittest.TestCase):_ _+_ _+ def testsimpleget(self):_ _+ a = AttrHolder()_ _+ a.foo = 100_ _+ a.attr42 = 1_ _+ self.assertEqual(a.('foo'), 100)_ _+ self.assertEqual(a.('fo' + 'o'), 100)_ _+ self.assertEqual(a.('f' + 'o' + [('o' if True else 'obar')][0]), 100)_ _+ self.assertEqual(a.({'FOO': 'fo'}['FOO'] + 'o'), 100)_ _+ self.assertEqual(a.('fo%s' % 'o'), 100)_ _+ self.assertEqual(a.('attr42'), 1)_ _+ self.assertEqual(a.('attr%d' % 42), 1)_ _+ self.assertEqual(a.('foo' if True else 'attr42'), 100)_ _+_ _+ def testnestedget(self):_ _+ a = AttrHolder()_ _+ a.b = AttrHolder()_ _+ a.b.c = 1_ _+ attrnameb = 'b'_ _+ attrnamec = 'c'_ _+ self.assertEqual(a.(attrnameb).(attrnamec), 1)_ _+_ _+ def testdefaultingget(self):_ _+ a = AttrHolder()_ _+ a.foo = 100_ _+ self.assertEqual(a.('foo', 99), 100)_ _+ self.assertEqual(a.('bar', 99), 99)_ _+ self.assertEqual(a.('baz', 99), 99)_ _+ self.assertEqual(a.('foo' if True else 'attr42', 99), 100)_ _+ self.assertEqual(a.('foo' if False else 'attr42', 99), 99)_ _+_ _+ @staticmethod_ _+ def attemptnonstringuse(attrname):_ _+ a = AttrHolder()_ _+ return a.(attrname)_ _+_ _+ def testonlystringsallowed(self):_ _+ self.assertRaises(TypeError, TestDynAttr.attemptnonstringuse, 99)_ _+ self.assertRaises(TypeError, TestDynAttr.attemptnonstringuse, None)_ _+ self.assertRaises(TypeError, TestDynAttr.attemptnonstringuse, 1.0)_ _+ self.assertRaises(TypeError, TestDynAttr.attemptnonstringuse, AttrHolder)_ _+ self.assertRaises(TypeError, TestDynAttr.attemptnonstringuse, sys)_ _+ self.assertRaises(TypeError, TestDynAttr.attemptnonstringuse, ())_ _+ self.assertRaises(TypeError, TestDynAttr.attemptnonstringuse, [])_ _+ self.assertRaises(TypeError, TestDynAttr.attemptnonstringuse, {})_ _+ self.assertRaises(TypeError, TestDynAttr.attemptnonstringuse, (1, 2))_ _+_ _+ def testaugassign(self):_ _+ a = AttrHolder()_ _+ a.foo = 100_ _+ a.('foo') += 10_ _+ self.assertEqual(a.foo, 110)_ _+ self.assertEqual(a.('fo' + 'o'), 110)_ _+ a.('f' + 'o' + 'o') *= 10_ _+ self.assertEqual(a.foo, 1100)_ _+ self.assertEqual(a.('fo' + 'o'), 1100)_ _+ a.('foobar'[:3]) /= 5_ _+ self.assertEqual(a.foo, 220)_ _+ self.assertEqual(a.('fo' + 'o'), 220)_ _+ a.(['foo', 'bar', 'baz'][0]) -= 40_ _+ self.assertEqual(a.foo, 180)_ _+ self.assertEqual(a.('fo' + 'o'), 180)_ _+_ _+ def testsetattr(self):_ _+ a = AttrHolder()_ _+ a.('foo') = 99_ _+ self.assertEqual(a.foo, 99)_ _+ a.('bar' + 'baz') = 100_ _+ self.assertEqual(a.barbaz, 100)_ _+_ _+ def testdelattr(self):_ _+ a = AttrHolder()_ _+ a.foo = 99_ _+ del a.('foo')_ _+ self.assertEqual(hasattr(a, 'foo'), False)_ _+_ _+_ _+def testmain():_ _+ testsupport.rununittest(TestDynAttr)_ _+_ _+if _name_ == "_main_":_ _+ testmain()_ _diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG_Python-2.5/Lib/test/testsyntax.py Python-2.5/Lib/test/testsyntax.py_ _--- ORIG_Python-2.5/Lib/test/testsyntax.py 2006-05-19 07:43:50.000000000 +0100_ _+++ Python-2.5/Lib/test/testsyntax.py 2007-02-02 17:21:04.991119000 +0000_ _@@ -235,6 +235,18 @@_ _>>> f() += 1 Traceback (most recent call last): SyntaxError: illegal expression for augmented assignment (<doctest test.testsyntax[33]>, line 1) + +Outlawed uses of dynamic attributes: + +>>> x.('foo', 0) += 1 +Traceback (most recent call last): +SyntaxError: augmented assignment to 2-argument dynamic-attribute expression not possible (<doctest test.testsyntax[34]>, line 1) +>>> x.('foo', 0) = 1 +Traceback (most recent call last): +SyntaxError: can't assign to 2-argument form of dynamic-attribute expression (<doctest test.testsyntax[35]>, line 1) +>>> del x.('foo', 0) +Traceback (most recent call last): +SyntaxError: can't delete 2-argument form of dynamic-attribute expression (<doctest test.testsyntax[36]>, line 1) """ import re diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG_Python-2.5/Parser/Python.asdl Python-2.5/Parser/Python.asdl --- ORIG_Python-2.5/Parser/Python.asdl 2006-04-04 05:00:23.000000000 +0100 +++ Python-2.5/Parser/Python.asdl 2007-02-02 12:39:36.665151000 +0000 @@ -72,6 +72,7 @@ -- the following expression can appear in assignment context | Attribute(expr value, identifier attr, exprcontext ctx) + | DynamicAttribute(expr value, expr attr, expr? dflt, exprcontext ctx) | Subscript(expr value, slice slice, exprcontext ctx) | Name(identifier id, exprcontext ctx) | List(expr* elts, exprcontext ctx) diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG_Python-2.5/Python/ast.c Python-2.5/Python/ast.c --- ORIG_Python-2.5/Python/ast.c 2006-09-05 04:56:01.000000000 +0100 +++ Python-2.5/Python/ast.c 2007-02-02 13:54:51.388446000 +0000 @@ -353,6 +353,13 @@ } e->v.Attribute.ctx = ctx; break; + case DynamicAttributekind: + if ((ctx == Store || ctx == Del) + && e->v.DynamicAttribute.dflt) + exprname = "2-argument form of dynamic-attribute expression"; + else + e->v.DynamicAttribute.ctx = ctx; + break; case Subscriptkind: e->v.Subscript.ctx = ctx; break; @@ -1427,7 +1434,7 @@ static exprty astfortrailer(struct compiling *c, const node *n, exprty leftexpr) { - /* trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME + /* trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME | '.' '(' test [',' test] ')' subscriptlist: subscript (',' subscript)* [','] subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop] */ @@ -1440,8 +1447,30 @@ return astforcall(c, CHILD(n, 1), leftexpr); } else if (TYPE(CHILD(n, 0)) == DOT ) { + if (TYPE(CHILD(n, 1)) == NAME) return Attribute(leftexpr, NEWIDENTIFIER(CHILD(n, 1)), Load, LINENO(n), n->ncoloffset, c->carena); + else { + exprty eval, edflt; + REQ(CHILD(n, 1), LPAR); + if (!(eval = astforexpr(c, CHILD(n, 2)))) + return NULL; + + if (TYPE(CHILD(n, 3)) == RPAR) { + assert(NCH(n) == 3); + edflt = NULL; + } else { + assert(NCH(n) == 5); + REQ(CHILD(n, 3), COMMA); + REQ(CHILD(n, 5), RPAR); + if (!(edflt = astforexpr(c, CHILD(n, 4)))) + return NULL; + } + + return DynamicAttribute(leftexpr, eval, edflt, Load, + LINENO(n), n->ncoloffset, + c->carena); + } } else { REQ(CHILD(n, 0), LSQB); @@ -1964,6 +1993,15 @@ case Attributekind: case Subscriptkind: break; + case DynamicAttributekind: + if (expr1->v.DynamicAttribute.dflt) { + asterror(ch, "augmented assignment to " + "2-argument dynamic-attribute " + "expression not possible"); + return NULL; + } + break; + default: asterror(ch, "illegal expression for augmented " "assignment"); diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG_Python-2.5/Python/ceval.c Python-2.5/Python/ceval.c --- ORIG_Python-2.5/Python/ceval.c 2006-08-13 19:10:10.000000000 +0100 +++ Python-2.5/Python/ceval.c 2007-02-08 12:56:39.637479000 +0000 @@ -1787,6 +1787,17 @@ if (err == 0) continue; break; + case STOREDYNAMICATTR: + w = POP(); + v = POP(); + u = POP(); + err = PyObjectSetAttr(v, w, u); /* v.w = u */ + PyDECREF(w); + PyDECREF(v); + PyDECREF(u); + if (err == 0) continue; + break; + case DELETEATTR: w = GETITEM(names, oparg); v = POP(); @@ -1795,6 +1806,14 @@ PyDECREF(v); break; + case DELETEDYNAMICATTR: + w = POP(); + v = POP(); + err = PyObjectSetAttr(v, w, (PyObject *)NULL); + PyDECREF(w); + PyDECREF(v); + break; + case STOREGLOBAL: w = GETITEM(names, oparg); v = POP(); @@ -1994,6 +2013,28 @@ if (x != NULL) continue; break; + case LOADDYNAMICATTR: + if (oparg) + u = POP(); + else + u = NULL; + w = POP(); + v = TOP(); + x = PyObjectGetAttr(v, w); + if (x == NULL && u != NULL + && PyErrExceptionMatches(PyExcAttributeError)) + { + PyErrClear(); + PyINCREF(u); + x = u; + } + PyDECREF(v); + PyDECREF(w); + PyXDECREF(u); /* This one may be NULL (if no default) */ + SETTOP(x); + if (x != NULL) continue; + break; + case COMPAREOP: w = POP(); v = TOP(); diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG_Python-2.5/Python/compile.c Python-2.5/Python/compile.c --- ORIG_Python-2.5/Python/compile.c 2006-08-12 02:45:47.000000000 +0100 +++ Python-2.5/Python/compile.c 2007-02-08 12:55:51.958809000 +0000 @@ -1357,6 +1357,13 @@ case SLICE+3: return -1; + case LOADDYNAMICATTR: + return -1 - oparg; + case STOREDYNAMICATTR: + return -3; + case DELETEDYNAMICATTR: + return -2; + case STORESLICE+0: return -2; case STORESLICE+1: @@ -3641,6 +3648,41 @@ return 0; } break; + case DynamicAttributekind: + { + int hasdefaultp = (e->v.DynamicAttribute.dflt != NULL); + + if (e->v.DynamicAttribute.ctx != AugStore) { + VISIT(c, expr, e->v.DynamicAttribute.value); + VISIT(c, expr, e->v.DynamicAttribute.attr); + if (hasdefaultp) + VISIT(c, expr, e->v.DynamicAttribute.dflt); + } + switch (e->v.DynamicAttribute.ctx) { + case AugLoad: + assert(!hasdefaultp); + ADDOPI(c, DUPTOPX, 2); + /* Fall through to Load */ + case Load: + ADDOPI(c, LOADDYNAMICATTR, hasdefaultp); + break; + case AugStore: + ADDOP(c, ROTTHREE); + /* Fall through to Store */ + case Store: + assert(!hasdefaultp); + ADDOP(c, STOREDYNAMICATTR); + break; + case Del: + ADDOP(c, DELETEDYNAMICATTR); + break; + default: + PyErrSetString(PyExcSystemError, + "invalid context in dynamic-attribute expression"); + return 0; + } + break; + } case Subscriptkind: switch (e->v.Subscript.ctx) { case AugLoad: @@ -3700,6 +3742,18 @@ auge->v.Attribute.ctx = AugStore; VISIT(c, expr, auge); break; + case DynamicAttributekind: + assert(e->v.DynamicAttribute.dflt == NULL); + auge = DynamicAttribute(e->v.DynamicAttribute.value, e->v.DynamicAttribute.attr, NULL, + AugLoad, e->lineno, e->coloffset, c->carena); + if (auge == NULL) + return 0; + VISIT(c, expr, auge); + VISIT(c, expr, s->v.AugAssign.value); + ADDOP(c, inplacebinop(c, s->v.AugAssign.op)); + auge->v.DynamicAttribute.ctx = AugStore; + VISIT(c, expr, auge); + break; case Subscriptkind: auge = Subscript(e->v.Subscript.value, e->v.Subscript.slice, AugLoad, e->lineno, e->coloffset, c->carena); diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG_Python-2.5/Python/symtable.c Python-2.5/Python/symtable.c --- ORIG_Python-2.5/Python/symtable.c 2006-08-12 02:43:40.000000000 +0100 +++ Python-2.5/Python/symtable.c 2007-02-08 12:53:36.279608000 +0000 @@ -1192,6 +1192,12 @@ case Attributekind: VISIT(st, expr, e->v.Attribute.value); break; + case DynamicAttributekind: + VISIT(st, expr, e->v.DynamicAttribute.value); + VISIT(st, expr, e->v.DynamicAttribute.attr); + if (e->v.DynamicAttribute.dflt) + VISIT(st, expr, e->v.DynamicAttribute.dflt); + break; case Subscriptkind: VISIT(st, expr, e->v.Subscript.value); VISIT(st, slice, e->v.Subscript.slice);

Python-ideas mailing list Python-ideas at python.org http://mail.python.org/mailman/listinfo/python-ideas

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-ideas mailing list