[Python-Dev] Modifying Grammar/grammar and other foul acts (original) (raw)

Georg Brandl g.brandl at gmx.net
Fri Mar 12 11:13:10 CET 2010

Am 09.03.2010 14:42, schrieb Jeremy Hylton:

On Sat, Mar 6, 2010 at 11:27 AM, Gregg Lind <gregg.lind at gmail.com> wrote:

Python-devs,

I'm writing to you for some help in understanding the Python grammar. As an excuse to deep dive into Python's tokenizer / grammar, I decided (as a hideous, hideous joke) to want to allow braces where colons are allowed (as flow control). Starting from PEP 306 (and branch r311), I hacked on Grammar/Grammer As a first example: funcdef: ('def' NAME parameters ['->' test] ':' suite | 'def' NAME parameters ['->' test] '{' suite '}' ) I reran Parser/pgen and the dfa changes, but python (3.1) when recompiled, throws errors on things like: def a() { None } Strangely enough: lambdef: ( 'lambda' [varargslist] ':' test | 'lambda' [varargslist] '{' test '}' ) works fine! I this simplely some difference between "test" and "suite". I have tried tackling this with gdb, looking at errinput clearly isn't enough. (gdb) break errinput (gdb) break PyParserASTFromString import sys b = compile("def a() {pass}","sys.stdout","single") # yet a simple grammar fix is enough for this! c = compile("lambda x {None}","sys.stdout","single") I'm in over my head! You don't say what errors occur when you try to compile strings in your new language. You may have changed the Grammar, which allows you to tokenize the input. That isn't enough to get the input to compile. You also need to change the compiler to understand the new tokens.

In particular, many AST creation functions check for specific counts of children on many nodes. I haven't checked, but in the case of the "funcdef" rule, it may check for either 7 or 5 children to determine whether the optional return annotation ['->' test] is present.

Georg

-- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.

More information about the Python-Dev mailing list