[Python-Dev] Move encoding_decl to the top of Grammar/Grammar? (original) (raw)

Thomas Lee tom at vector-seven.com
Tue Dec 2 09:00:16 CET 2008

Here's the corresponding tracker issue:

http://bugs.python.org/issue4347

I've uploaded a patch there anyway, since I'm going to need this stuff working for a presentation I'm giving tomorrow.

Cheers, T

Thomas Lee wrote:

Hi all,

Currently, Parser/parsetok.c has a dependency on graminit.h. This can cause headaches when rebuilding after adding new syntax to Grammar/Grammar because parsetok.c is part of pgen, which is responsible for generating graminit.h. This circular dependency can result in parsetok.c using a different value for encodingdecl to what is used in ast.c, which causes PyASTFromNode to fall over at runtime. It effectively looks something like this: * Grammar/Grammar is modified * build begins -- pgen compiles, parsetok.c uses encodingdecl=X * graminit.h is rebuilt with encodingdecl=Y * ast.c is compiled using encodingdecl=Y * when python runs, parsetok() emits encodingdecl nodes that PyASTFromNode can't recognize: SystemError: invalid node XXX for PyASTFromNode A nice, easy short term solution that doesn't require unwinding this dependency would be to simply move encodingdecl to the top of Grammar/Grammar and add a big warning noting that it needs to come before everything else. This will help to ensure its value never changes when syntax is added/removed. I'm happy to provide a patch for this (including some additional dependency info for files dependent upon graminit.h and Python-ast.h), but was wondering if there were any opinions about how this should be resolved. Cheers, Tom

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com

More information about the Python-Dev mailing list