[Python-3000] Refactoring tool available (work in progress) (original) (raw)

Talin talin at acm.org
Fri Dec 15 07:56:49 CET 2006

Guido van Rossum wrote:

In the sandbox I've been working on a refactoring tool, which could form the basis for a Python 2.x -> 3.0 conversion tool. I'd like to invite folks here to give it a try and give me a hand. It certainly needs more work, but I think that the basic infrastructure is sound. Check out sandbox/2to3/: http://svn.python.org/view/sandbox/trunk/2to3/.

This message is to invite feedback, and to encourage contributions. It would be great if people tried their hands at writing new transformations!

And here I thought Py3K wasn't going to have programmable syntax :)

I was working on a related idea recently, involving syntactical transformations, although not directly related to Python. One of the things I discovered was that it was possible in many cases to use the same parser on both the transformation patterns and the text to be translated. This allowed the transformation patterns to look a lot like the source and target patterns, instead of having to have a special syntax for the transformations.

Ideally, you ought to be able to say something like:

apply(?f, ?x, ?y) => f(*x, **y)

You'd have to add an exception to the parser to recognize the unbound variable prefix ('?') when parsing patterns.

You could also put constraints on the bound variables:

apply(?(expr:f), ?x, ?y) => (f)(*x, **y)
apply(?f, ?x, ?y) => f(*x, **y)

In other words, if we need parens (because 'f' is an expression), then match the first rule, otherwise the second. Although a better way to handle the case of the parens is by repeated transformation:

apply(?f, ?x, ?y) => paren(f)(*x, **y)
paren(?(var:f)) => f
paren(?f) => (f)

In this case, we transform 'f' into 'paren(f)', and then transform that into either 'f' or '(f)' depending on whether it's a simple variable or an expression. This allows us to only have one version of the 'apply' rule.

(Although the specific case of parenthesis logic ought to be built in, as you have already done, since it's going to clutter up every rule otherwise. Mainly I wanted to use it as an example to show constraints.)

What I'm describing is exactly the same logic as an algebraic solver, which many people have written in Python already.

One other thing you might want to do is look at the parser generator ANTLR (www.antlr.org), which has a syntax for tree transformations and pattern matching of tree fragments.

Anyway - it looks pretty cool :)

-- Talin

More information about the Python-3000 mailing list