[Python-Dev] Where are universal newlines handled in the parser/compiler? (original) (raw)
Brett Cannon brett at python.org
Sun Aug 17 03:34:43 CEST 2008
- Previous message: [Python-Dev] eval() accepts compiled 'exec' statements
- Next message: [Python-Dev] Where are universal newlines handled in the parser/compiler?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
If you import a module that uses '\r\n' line endings, Python does the right thing. But if you read in the bytes for the same file and then pass it to compile() you get an unhelpful syntax error pointing at a blank line.
Normally I would say one should just open the source file as 'r' instead of 'rb', but with source code that does not work well as their can be a source encoding set. Lib/test/test_pep263.py is the perfect example of this; Windows newlines with a koi8-r encoding.
What I would like to do is get compile() to work properly with a bytes stream just as if Python itself was handling the compilation through import and from a file directly. But before I try to dig into the parser to figure out where the translation of newlines occurs (or where the translation option is set), I thought I would ask to see if anyone just happened to know (I have already spent a few hours figuring out why Latin-1 encodings were not working with compile() so I don't want to go diving into the maze of function calls in the parser again).
-Brett
- Previous message: [Python-Dev] eval() accepts compiled 'exec' statements
- Next message: [Python-Dev] Where are universal newlines handled in the parser/compiler?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]