Issue 33477: Document that compile(code, 'exec') has different behavior in 3.7+ (original) (raw)

In recent Python the following

from ast import PyCF_ONLY_AST compile("'a'", 'whatever', 'exec', PyCF_ONLY_AST).body

In 3.6 it return

[<_ast.Expr at 0x10b7441d0>] # that contail Str('a')

While on master:

[]

This is inconveninent for alternative repl like IPython, where basically if the user is entering a single string, the result is nothing in Python 3.7+, while it does return something on earlier Python [1].

The documentation of compile says:

... it can be 'exec' if source consists of a sequence of statements,

Which is not technically true any more as the first statement, if a string, will be removed.

What's happening here is that since Python 3.7 if the first statement is actually an expression containing a lonely string it is assign to the module docstring. So that's basically assuming you are parsing a module, and that the docstring make sens in this context, while in a REPL you are parsing a sucesssion of statements, in which case there is no need for a docstring that make no sens in this context.

This is usually not an issue, unless this lonely statement is also the last, and what the user wants to execute in a REPL, in which case it has no side effect.

I don't have any objection to the new behavior, though I was wondering if this kind of side effect was anticipated.

If that affect IPython, it will likely effect other alternative REPLs.

Thus, I believe it would be good to have this at least documented a tiny bit better and added in what's new, and potentially clarified in the exec docs.

I could argue that now the exec name may be a tiny bit unsuitable for the new behavior, and would love if this could be optional.

1: https://github.com/ipython/ipython/issues/11133#issuecomment-388591332

Fair enough that what's new include things about Module

The first statement in their body is not considered as a docstring anymore.

Note that this sentence read backward to me. I understand what is meant because I know the new behavior. It might be good to clarify. potentially:

The first statement in the body attribute of should not be considered the docstring of the module anymore, the docstring attribute is reserved for that.

Though the documentation of compile() does not say that compile(...,'exec') compile a module. It says:

The mode argument specifies what kind of code must be compiled; it can be 'exec' if source consists of a sequence of statements

Which now is incorrect. I was expecting compile(..., 'exec') to return a Module with None or empty string as the docstring attribute – which is also a perfectly reasonable request.

I think that compile documentation should be changed to reflect what it does.

Or (but I see why this is un-reasonable) split add the mode='module' that has new behavior, while exec does not.