Issue 19080: Enrich SyntaxError with additional information (original) (raw)

Created on 2013-09-23 20:42 by alonho, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (4)
msg198341 - (view)	Author: alon horev (alonho) *	Date: 2013-09-23 20:42
Some context for this feature request: I'm using the wonderful ast module for a library that translates python code to MongoDB queries (https://github.com/alonho/pql). I also did the same for SQL queries using sqlalchemy as a part of another project (https://github.com/alonho/pytrace). One of the things I find lacking in python's parser is additional information about SyntaxErrors. This could help users of the 'ast' module, IDE and developers. Here are some examples of what I'd like to see 1. ast.parse('* 2') -> SyntaxError('Unexpected operator at start of an expression') 2. ast.parse('2 *') -> SyntaxError('Missing right hand side operand') 3. ast.parse('if a = 1: pass') -> SyntaxError('Cannot assign inside an expression') There are several challenges here: 1. Does the parser have this information and doesn't surface it? 2. Can such messages be automatically generated without filling the code with error handling code? 3. Which part of the code could be responsible for this kind of a task? I've looked at the BNF and it contains more than just syntax legality but operator precedence and such. Perhaps there's another (simpler) grammar definition somewhere? I was curious to see what Ruby does, and it uses a simple solution of providing raw information along with the exception: >> a == * 1 SyntaxError: compile error (irb):17: syntax error, unexpected tSTAR a == * 1 ^ from (irb):17
msg198555 - (view)	Author: Terry J. Reedy (terry.reedy) *	Date: 2013-09-28 21:11
Currently, >>> * 2 # or * a SyntaxError: can use starred expression only as assignment target This conflicts with your desired message. It cannot change because *a, at least, is a valid statement prefix in a way that '/ a' is not. Most other things give an uninformative generic 'invalid syntax' message (plus a more informative location indicator). If you can find a way to improve this, lots of people would be happy ;-).
msg356928 - (view)	Author: Batuhan Taskaya (BTaskaya) *	Date: 2019-11-18 22:59
I am not sure about how to implement this to the core, but you can combine tokenize with ast to reproduce this kind of messages in your projects. You can match tokens with SyntaxErrors by lineno and offset.
msg390110 - (view)	Author: Terry J. Reedy (terry.reedy) *	Date: 2021-04-02 22:51
I am closing because 2 of your 3 examples (and many others) have had the messages changed in an effort to be more informative. But Pablo can reverse this if he wants. In 3.10: >>> * 2 SyntaxError: can't use starred expression here +2 and -2 are also valid expressions. /2 is not and still gets the old message. >>> 2 * SyntaxError: invalid syntax # Same. >>> if a=1: pass SyntaxError: expected ':' # At '=', which *is* an error. Also not what you you suggested, which points to the impossibility of guessing what the coder meant.