bpo-33260: Regenerate token.py after removing ASYNC and AWAIT. (GH-6447) · python/cpython@d08972f (original) (raw)

Skip to content

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Appearance settings

Commit d08972f

File tree

3 files changed

lines changed

3 files changed

lines changed

Original file line number Diff line number Diff line change
@@ -172,7 +172,7 @@ Common syntax elements for comprehensions are:
172 172
173 173 .. productionlist::
174 174 comprehension: `expression` `comp_for`
175 - comp_for: [ASYNC] "for" `target_list` "in" `or_test` [`comp_iter`]
175 + comp_for: ["async"] "for" `target_list` "in" `or_test` [`comp_iter`]
176 176 comp_iter: `comp_for` | `comp_if`
177 177 comp_if: "if" `expression_nocond` [`comp_iter`]
178 178
Original file line number Diff line number Diff line change
@@ -65,14 +65,12 @@
65 65 ELLIPSIS = 52
66 66 # Don't forget to update the table _PyParser_TokenNames in tokenizer.c!
67 67 OP = 53
68 -AWAIT = 54
69 -ASYNC = 55
70 -ERRORTOKEN = 56
68 +ERRORTOKEN = 54
71 69 # These aren't used by the C tokenizer but are needed for tokenize.py
72 -COMMENT = 57
73 -NL = 58
74 -ENCODING = 59
75 -N_TOKENS = 60
70 +COMMENT = 55
71 +NL = 56
72 +ENCODING = 57
73 +N_TOKENS = 58
76 74 # Special definitions for cooperation with parser
77 75 NT_OFFSET = 256
78 76 #--end constants--
Original file line number Diff line number Diff line change
@@ -284,7 +284,7 @@ def compat(self, token, iterable):
284 284 self.encoding = tokval
285 285 continue
286 286
287 -if toknum in (NAME, NUMBER, ASYNC, AWAIT):
287 +if toknum in (NAME, NUMBER):
288 288 tokval += ' '
289 289
290 290 # Insert a space between two consecutive strings