[Python-Dev] Reading Python source file (original) (raw)

Guido van Rossum guido at python.org
Thu Nov 19 13:27:18 EST 2015

On Thu, Nov 19, 2015 at 3:51 AM, Serhiy Storchaka <storchaka at gmail.com> wrote:

On 17.11.15 18:50, Guido van Rossum wrote:

On Tue, Nov 17, 2015 at 8:20 AM, Serhiy Storchaka <storchaka at gmail.com> wrote:

Current implementation of import system went the same way. As a result importing the script as a module and running it with command line can have different behaviours in corner cases. I'm confused. Of course these two behaviors differ, since Python uses a different name. Not sure how this relates to the REPL. Sorry for confusing. I meant parser level. File parser has a few bugs, that can cause that the source will be differently interpreted with file and string parsers. For example attached script produces different output, "ä" if executed as a script, and "À" if imported as a module.

I see. Well, I trust you in this area (it's been too long since I wrote the original code for all that :-).

And there is a question about the null byte. Now compile(), exec(), eval() raises an exception if the script contains the null byte. Formerly they accepted it, but the null byte ended the script.

I like erroring out better.

The behavior of file parser is more weird. The null byte makes parser to ignore the end of script including the newline byte [1]. E.g. "#\0\nprint('a')" is interpreted as "#print('a')". This is different from PyPy (and may be other implementations) that interprets the null byte just as ordinal character.

Yeah, this is just poorly written old code that didn't expect anyone to care about null bytes. IMO here too the null byte should be an error.

The question is wherever we should support the null byte in Python sources.

[1] http://bugs.python.org/issue20115

Good luck, and thanks for working on this!

-- --Guido van Rossum (python.org/~guido)

More information about the Python-Dev mailing list