[Python-3000] Refactoring tool available (work in progress) (original) (raw)

Guido van Rossum guido at python.org
Fri Dec 15 18:52:08 CET 2006

I'm taking the idea of using the same parser with a grain of salt -- by the time you have added all the pattern matching syntax IMO it will look too cluttered. Also, your concept of "variable" is too constrained -- the parenthesization should not only depend on the form of the experssion, but also on the context. E.g. "if x.has_key(a+b):" can translate to "if a+b in x:", but "3x.has_key(a+b)" has to translate to "3(a+b in x)".

I'm not sure how thinking about it as an algebraic solver helps -- is there any existing code I can borrow?

I looked at this bit of ANTLR briefly: http://www.antlr.org/doc/sor.html#_bb8; from the example it looks like their notation isn't any more compact than the same thing written in Python (despite ANTLR's over-use of line-noise characters). In fact, I can probably write that example using my tool easily: the pattern is something like

arith_expr<t=term '+' n=NUMBER | n=NUMBER '+' t=term>

and the transformation (after setting up t and n to point to the respective subtrees) could be something like

try: value = int(n) except ValueError: return None if value != 0: return None return t.clone()

--Guido

On 12/14/06, Talin <talin at acm.org> wrote:

Guido van Rossum wrote: > In the sandbox I've been working on a refactoring tool, which could > form the basis for a Python 2.x -> 3.0 conversion tool. I'd like to > invite folks here to give it a try and give me a hand. It certainly > needs more work, but I think that the basic infrastructure is sound. > Check out sandbox/2to3/: > http://svn.python.org/view/sandbox/trunk/2to3/. > > This message is to invite feedback, and to encourage contributions. It > would be great if people tried their hands at writing new > transformations!

And here I thought Py3K wasn't going to have programmable syntax :) I was working on a related idea recently, involving syntactical transformations, although not directly related to Python. One of the things I discovered was that it was possible in many cases to use the same parser on both the transformation patterns and the text to be translated. This allowed the transformation patterns to look a lot like the source and target patterns, instead of having to have a special syntax for the transformations. Ideally, you ought to be able to say something like: apply(?f, ?x, ?y) => f(*x, **y) You'd have to add an exception to the parser to recognize the unbound variable prefix ('?') when parsing patterns. You could also put constraints on the bound variables: apply(?(expr:f), ?x, ?y) => (f)(*x, **y) apply(?f, ?x, ?y) => f(*x, **y) In other words, if we need parens (because 'f' is an expression), then match the first rule, otherwise the second. Although a better way to handle the case of the parens is by repeated transformation: apply(?f, ?x, ?y) => paren(f)(*x, **y) paren(?(var:f)) => f paren(?f) => (f) In this case, we transform 'f' into 'paren(f)', and then transform that into either 'f' or '(f)' depending on whether it's a simple variable or an expression. This allows us to only have one version of the 'apply' rule. (Although the specific case of parenthesis logic ought to be built in, as you have already done, since it's going to clutter up every rule otherwise. Mainly I wanted to use it as an example to show constraints.) What I'm describing is exactly the same logic as an algebraic solver, which many people have written in Python already. One other thing you might want to do is look at the parser generator ANTLR (www.antlr.org), which has a syntax for tree transformations and pattern matching of tree fragments. Anyway - it looks pretty cool :) -- Talin

Python-3000 mailing list Python-3000 at python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-3000 mailing list