[Python-Dev] Policy for making changes to the AST (original) (raw)
Eugene Toder eltoder at gmail.com
Sun Apr 3 03:55:45 CEST 2011
- Previous message: [Python-Dev] not possible to install python 3.2
- Next message: [Python-Dev] Policy for making changes to the AST
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hello,
While working on a rewrite of peephole optimizer that works on AST (http://bugs.python.org/issue11549) I had to make a few changes to the structure of AST. The changes are described in the issue. This raises the question -- what is the policy for making changes to the AST? Documentation for ast module does not warn about possible changes, but obviously changes can occur, for example, when new constructs are introduced. What about other changes? Is there a policy for what's acceptable and how this should be handled?
Assuming we do want to make changes (not just extensions for new language features), I see 3 options for handling them:
Do nothing. This will break code that currently uses AST, but doesn't add any complexity to cpython.
Write a pure-Python compatibility layer. This will convert AST between old and new formats, so that old code continues working. To do this a) Introduce ast.compile function (similar to ast.parse), which should be the recommended way of compiling to AST. b) Add ast_version argument to ast.parse and ast.compile defaulting to 1. c) Preserve old AST node classes and attributes in Python. d) If ast_version specified is not the latest, ast.parse and ast.compile would convert from/to latest version in Python using ast.NodeTransformer.
This is not fully backward compatible, but allows to do all staging in Python.
- Full backward compatibility (with Python code). This means conversion is done in compile(). It can either call Python conversion code from ast module, or actually implement conversion in C using AST visitors. Using my visitors generator this should not be very hard. Downsides here are a lot of C code and no clear separation of deprecated AST nodes (they will remain in Python.asdl). Otherwise it's similar to 2, with ast_version argument added to compile() and ast.parse.
For 2 and 3 we can add a PendingDeprecationWarning when ast_version 1 is used.
In any case, C extensions that manipulate AST will be broken, but 3 provides a simple way to update them -- insert calls to C conversion functions.
Eugene
- Previous message: [Python-Dev] not possible to install python 3.2
- Next message: [Python-Dev] Policy for making changes to the AST
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]