[Python-Dev] astoptimizer: static optimizer working on the AST (original) (raw)

Victor Stinner victor.stinner at gmail.com
Tue Mar 26 22:56:53 CET 2013

Hi,

I made progress since last August on my astoptimizer project (read the Changelog). Previous email thread: http://mail.python.org/pipermail/python-dev/2012-August/121286.html

The astoptimizer project is an optimizer rewriting Python AST. It executes as much code as possible during the compilation. The optimizer itself is not designed to be fast, but to emit faster code.

https://bitbucket.org/haypo/astoptimizer/

Some optimizations are not "pythonic": don't respect the Python language by making some assumptions (ex: on the namespace), which may be wrong in some cases. astoptimizer is written for adults which know which optimizations can be enabled, and which ones must be disabled in their application (application, not module). I'm trying to write a safe and "pythonic" default configuration. For example, len("abc") is not replaced with 3 by default. You have to enable explicitly the "builtin_funcs" configuration feature.

astoptimizer can be used as a pythonic preprocessor: it replaces os.name, sys.platform and also your own constants by their value. It removes dead code and so may be used to remove completly the overhead of checks on the Python version or an the platform (if python3: ... else: ...).

I would like to improve the integration of astoptimizer in Python 3.4. Brett Canon asked me to use an hook in importlib, I proposed to add a generic AST hook which would also be called by eval(), compile(), etc. I would like to allow anyone to use its own AST modifier, and simplify the usage of astoptimizer. => http://bugs.python.org/issue17515

Open issues:

--

There are many open issues proposing to write a better Python optimizer. Some of them are stuck because implementing them using the bytecode optimizer is hard.

astoptimizer implements many optimizations listed in these issues. Read the README file to the list of optimizations already implemented, and the TODO file for ideas of new optimizations.

--

The project is still experimental. I ran Python 2.7 and 3.4 test suites. Some tests are failing because the AST or the bytecode is different (which is expected). More tests are failing if you enable more agressive optimizations (especially if you enable the agressive mode to remove dead code). At least, Python does not crash :-) (It occurs sometimes when astoptimizer generates invalid AST!)

Victor

More information about the Python-Dev mailing list