Issue 18965: 2to3 can produce illegal bytes literals (original) (raw)

This issue has been migrated to GitHub: https://github.com/python/cpython/issues/63165

classification

Title: 2to3 can produce illegal bytes literals
Type: behavior Stage: resolved
Components: 2to3 (2.x to 3.x conversion tool) Versions: Python 3.4, Python 3.5, Python 2.7

process

Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, r.david.murray, serhiy.storchaka, vinay.sajip
Priority: normal Keywords: patch

Created on 2013-09-08 00:06 by serhiy.storchaka, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
2to3_nonascii_bytes-2.7.patch serhiy.storchaka,2013-10-03 10:46 review
Messages (9)
msg197205 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-09-08 00:06
In Python 2 string literals are allowed to contain non-ascii characters. In Python 3 bytes literals can contains only ascii characters, non-ascii characters should be escaped with \xXX. However 2to3 doesn't change b'...' literals and produces illegal code. I propose new fixer which escapes non-ascii bytes in bytes literals. Here is a patch (without tests yet).
msg198888 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-10-03 10:21
Added a test.
msg198889 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-10-03 10:46
Backported to 2.7.
msg199861 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-10-14 10:48
Benjamin, what would you say about this?
msg199893 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2013-10-14 14:18
I think it's too bad parsing of b-prefixed strings in Python 2 doesn't reject non-ASCII characters.
msg231657 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-11-25 12:51
Now Python 2 with the -3 option warns about b-prefixed strings with non-ASCII characters ().
msg240361 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-04-09 17:44
Does the addition of the warning in python3 make the fixer obsolete? In other words, should we close this issue?
msg240362 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-09 18:06
The addition of the warning in python3 is not related to this issue, because produced code even is not compiled in Python 3. But the addition of the warning in Python 2 made this issue less important.
msg240376 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-04-09 20:14
Sorry, I meant the addition of the -3 warning.
History
Date User Action Args
2022-04-11 14:57:50 admin set github: 63165
2015-04-09 20:14:13 r.david.murray set messages: +
2015-04-09 18:06:08 serhiy.storchaka set status: open -> closedresolution: rejectedmessages: + stage: patch review -> resolved
2015-04-09 17:44:25 r.david.murray set nosy: + r.david.murraymessages: +
2014-11-25 12:51:46 serhiy.storchaka set messages: + versions: + Python 3.5, - Python 3.3
2013-10-14 14🔞00 benjamin.peterson set messages: +
2013-10-14 10:48:40 serhiy.storchaka set messages: +
2013-10-03 10:46:07 serhiy.storchaka set files: + 2to3_nonascii_bytes-2.7.patchmessages: +
2013-10-03 10:45:32 serhiy.storchaka set files: - 2to3_nonascii_bytes.patch
2013-10-03 10:21:20 serhiy.storchaka set files: + 2to3_nonascii_bytes.patchmessages: +
2013-10-03 10:20:02 serhiy.storchaka set files: - 2to3_nonascii_bytes.patch
2013-09-13 17:49:59 terry.reedy set nosy: + benjamin.peterson
2013-09-08 00:06:39 serhiy.storchaka create