Python lint tool (was: Re: [Python-Dev] Python keywords (was Lockstep iteration (original) (raw)

James C. Ahlstrom jim@interet.com
Mon, 14 Aug 2000 12:29:08 -0400

Trent Mick wrote:

On Mon, Aug 14, 2000 at 11:25:59AM -0400, James C. Ahlstrom wrote: > What about using Bison/Yacc? I have been playing with a > lint tool for Python, and have been using it. > Oh yeah? What does the linter check? I would be interested in seeing that.

Actually I have better luck parsing Python than linting it. My initial naive approach using C-language wisdom such as checking for line numbers where variables are set/used failed. I now feel that a Python lint tool must either use complete data flow analysis (hard) or must actually interpret the code as Python does (hard). All I can really do so far is get and check function signatures. I can supply more details if you want, but remember it doesn't work yet, and I may not have time to complete it. I learned a lot though.

To parse Python I first use Parser/tokenizer.c to return tokens, then a Yacc grammar file. This parses all of Lib/*.py in less than two seconds on a modest machine. The tokens returned by tokenizer.c must be massaged a bit to be suitable for Yacc, but nothing major.

All the Yacc actions are calls to Python methods, so the real work is written in Python. Yacc just represents the grammar.

The problem I have with the current grammar is the large number of confusing shifts required. The grammar can't specify operator precedence, so it uses shift/reduce conflicts instead. Yacc eliminates this problem.

JimA