Issue 33991: lib2to3 should parse f-strings (original) (raw)

Currently f-strings are parsed just as regular strings by lib2to3. However, this has the problem that the invariant of a string node being a literal is now broken.

This poses two problems. On one hand, if I want to compare that two string nodes are equivalent I would need to do something like:

def string_nodes_are_eqivalent(node1, node2): if is_f_string(node1) and is_f_string(node2): # This would require to parse the nodes using ast.parse return f_strings_are_equivalent(node1, node2) if not is_f_string(node1) and not is_f_string(node2): return ast.literal_eval(node1.value) == ast.literal_eval(node2.value) return False

Note that ast.literal_eval does not accept f-strings. Also note that ast.parse returns an ast.JoinedString for f-strings, whose elements are either ast.Str or ast.FormattedValue, the latter being ast expressions.

On the other hand, this has the problem of not being able to refactor the expressions inside an f-string.

Note also, that lib2to3 will parse invalid f-strings like f"hello {", whereas ast.parse will raise a SyntaxError exception.

See below for reproduction cases: In [2]: lib2to3.tests.support.parse_string('f"hello {"') Out[2]: Node(file_input, [Node(simple_stmt, [Leaf(3, 'f"hello {"'), Leaf(4, '\n')]), Leaf(0, '')])

In [4]: ast.parse('f"hello {"') Traceback (most recent call last):

File "/Users/skreft/.virtualenvs/pyfaster/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2963, in run_code exec(code_obj, self.user_global_ns, self.user_ns)

File "", line 1, in ast.parse('f"hello {"')

File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ast.py", line 35, in parse return compile(source, filename, mode, PyCF_ONLY_AST)

File "", line 1 SyntaxError: f-string: expecting '}'