unparse.py does not handle u"bar" correctly (original) (raw)

Currently (135c6a56e55d2f4f8718b3b9f03ce3c692b15f0f) the following test fails:

./python -m test.regrtest -v test_tools.test_unparse -u cpu

(Note that -u cpu is needed to force test_unparse.DirectoryTestCase to check all files under Lib/ and Lib/test/.)

An example error message:

====================================================================== FAIL: test_files (test.test_tools.test_unparse.DirectoryTestCase) (filename='/home/yen/Projects/cpython/Lib/test/test_typing.py' )

Traceback (most recent call last): File "/home/yen/Projects/cpython/Lib/test/test_tools/test_unparse.py", line 309, in test_files self.check_roundtrip(source) File "/home/yen/Projects/cpython/Lib/test/test_tools/test_unparse.py", line 132, in check_roundtrip self.assertASTEqual(ast1, ast2) File "/home/yen/Projects/cpython/Lib/test/test_tools/test_unparse.py", line 124, in assertASTEqual self.assertEqual(ast.dump(ast1), ast.dump(ast2)) AssertionError: 'Modu[88178 chars]kind='u')], ctx=Load())), ctx=Load()))], dec[421987 chars]=[])' != 'Modu[88178 chars]kind=No ne)], ctx=Load())), ctx=Load()))], deco[421986 chars]=[])'

Apparently that's because Tools/parser/unparse.py does not handle strings like u"bar" correctly. I managed to "fix" it with the following patch:

diff --git a/Tools/parser/unparse.py b/Tools/parser/unparse.py index 385902ef4b..a25b5e49df 100644 --- a/Tools/parser/unparse.py +++ b/Tools/parser/unparse.py @@ -399,6 +399,8 @@ class Unparser: elif value is ...: self.write("...") else:

```
       if t.kind == 'u':
```

           self.write("u")
       self._write_constant(t.value)

def _List(self, t):

Not sure if this is the correct approach, though.

Confirmed that this fixes the broken buildbots:

./python -m test test_tools -vvvv -uall -m test_files == CPython 3.8.0a4+ (heads/master:91f4380ced, May 26 2019, 17:30:21) [GCC 8.3.0] == Linux-5.0.15-1-MANJARO-x86_64-with-glibc2.29 little-endian == cwd: /home/pablogsal/github/cpython/build/test_python_25649 == CPU count: 12 == encodings: locale=UTF-8, FS=utf-8 Run tests sequentially 0:00:00 load avg: 0.50 [1/1] test_tools test_files (test.test_tools.test_unparse.DirectoryTestCase) ... Testing /home/pablogsal/github/cpython/Lib/typing.py Testing /home/pablogsal/github/cpython/Lib/test/test_typing.py ok

Ran 1 test in 0.586s

== Tests result: SUCCESS ==

1 test OK.

Total duration: 671 ms Tests result: SUCCESS

The manual check is needed because the CI in GitHub does not run test_tools over all files so it does not take forever.