[Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer) (original) (raw)

Eugene Toder eltoder at gmail.com
Sat Mar 12 06:39:55 CET 2011

Experience shows that optimizations are always error prone, no matter what framework or internal representation you use. On that basis, I believe that we ought to declare peephole.c as being somewhat off-limits for further development (except for small adaptations if the underlying opcodes change meaning).

With this attitude, it would be more logical to remove the whole thing completely. Even if you don't touch the optimizer, any changes in the compiler can start producing code that exposes a previously hidden bug.

The bright ideas for other enhancements aren't actually new ideas. They were considered at the outset and not included for a reason.

It would help if these were documented.

It would be more correct

How can you be more correct than tautological truth? :)

1) much of the semantic information available the AST has been lost by the time the bytecode is generated

Not really. Many systems use stack based bytecode as an intermediate form and reconstruct different IRs from it. Examples are JVM, CLR and PyPy. The real problem with bytecode is that it's a representation optimized for storage and interpretation. Transforming it is harder, less efficient and more error-prone than transforming a tree structure. But transformations of trees are in no way automatically bug free.

2) peephole.c is having to disassemble the bytecode, reconstruct semantic relationships as well as it can, recognize transformation patterns and then regenerate bytecode.

v8 does peephole directly on x86 machine code. No kittens were harmed.

Now, I do agree with you that moving optimizations to AST is a step forward. Moving them to a specialized IR in SSA form would be even better. If there was a framework for AST-based optimizations in the current code, I'd write my code to use it. However, I couldn't find it.

What I don't understand is why doing 90% of the work and refuse to do the last 10%. Looking at the actual patches, I do not see significant increase in complexity or code size -- all the complex cases are already handled by existing code. In some cases the code became cleaner and simpler.

Not really.  It is darned difficult to design tests to catch all the corner cases.

You either do that or do not write optimizations. No one said it should be easy.

You've got wishful thinking if you think a handful of tests can catch errors in code that sophisticated.

Why limit yourself with a handful of tests? Python is widespread, there's a lot of code in Python. Unlike with libraries, any code you run tests the optimizer, so just run a lot of code. And, as I've said, write a test generator.

Cheers, Eugene

More information about the Python-Dev mailing list