Issue 23466: PEP 461: Inconsistency between str and bytes formatting of integers (original) (raw)

Created on 2015-02-15 18:21 by serhiy.storchaka, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
bytes_int_format.patch serhiy.storchaka,2015-02-16 21:50 review
bytes_format_overflow.patch serhiy.storchaka,2015-04-03 06:04 review
Messages (18)
msg236056 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-15 18:21
PEP 461 says that all numeric bytes formatting codes will work as they do for str. In particular b"%x" % val is equivalent to ("%x" % val).encode("ascii"). But this is wrong with current implementation: >>> '%x' % 3.14 Traceback (most recent call last): File "", line 1, in TypeError: %x format: an integer is required, not float >>> b'%x' % 3.14 b'3' The same is for %X, %o and %c. Raising TypeError on non-integer input to %c, %o, %x, and %X was added in .
msg236113 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-16 21:50
Here is a patch. The difference between string and bytes formatting is that %c with out of the range integer raises OverflowError for str and TypeError for bytes. May be ValueError is more suitable in both cases.
msg239157 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-24 16:37
Ethan?
msg239191 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-24 22:22
The patch looks good to me, except of a question added on the review.
msg239553 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-03-30 01:51
Patch looks good. Changing the raised exceptions to ValueError would require deprecation periods.
msg239561 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-30 02:22
> Changing the raised exceptions to ValueError would require deprecation periods. bytes%args is not a new feature of python 3.5? It sounds strange to deprecate a part of a new feature.
msg239569 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-30 06:22
New changeset cb96fd376baa by Serhiy Storchaka in branch 'default': Issue #23466: %c, %o, %x, and %X in bytes formatting now raise TypeError on https://hg.python.org/cpython/rev/cb96fd376baa
msg239922 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-04-02 15:20
It's a new feature for 3.5 that is partly responsible for compatibility with 2.7 code. 2.7 raises Overflow error, so 3.5 should also (for out of range values -- wrong value types raise TypeError).
msg239933 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-04-02 17:30
b'%c' is still raising a TypeError. The error message is fine ("%c requires an integer in range(256) or a single byte") but it should be an OverflowError for backwards compatibility.
msg239941 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-02 18:30
OverflowError is for platform limitations (such as the size of machine word or addressed space). When limits are well defined and platform-independent, ValueError or may be TypeError are considered as better types. It would be better to change OverflowError to ValueError or TypeError in formatting.
msg239955 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-03 06:04
Here is a patch that makes bytes formatting raise an OverflowError if integer argument of %c is out of range.
msg239958 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-04-03 08:48
> b'%c' is still raising a TypeError. The error message is fine ("%c requires an integer in range(256) or a single byte") but it should be an OverflowError for backwards compatibility. I don't understand why you care so much on the exact exception. It doesn't look right to me to rely on the *exact* exception raised by "%c" % arg. It's an obvious bug in the application. Sometimes, you may want to be extra safe and catch exception while formating a message. The logging module does this. But the logging doesn't care of the exact exception, it uses a generic "except Except:" in StreamHandler.emit(): def emit(self, record): try: msg = self.format(record) stream = self.stream stream.write(msg) stream.write(self.terminator) self.flush() except Exception: self.handleError(record) IMO b"%c" % int must raise ValueError, not OverflowError, if the value is not in the range 0..255.
msg239962 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-03 09:10
See also .
msg240011 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2015-04-03 17:01
Looks good, thanks Serhiy.
msg240018 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-04-03 17:54
New changeset 313fd1c819c5 by Serhiy Storchaka in branch 'default': Issue #23466: Raised OverflowError if %c argument is out of range. https://hg.python.org/cpython/rev/313fd1c819c5
msg240070 - (view) Author: Wolfgang Maier (wolma) * Date: 2015-04-04 14:07
the new test: test_exc('%x', '1', TypeError, "%x format: a number is required, not str") expects the wrong error message. python -m unittest -v test.test_format ... '%x' % '1' works? ... no Unexpected <class 'TypeError'> : '%x format: an integer is required, not str' ... -> it's "an integer", not "a number"
msg240077 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-04-04 14:30
New changeset 11e6986c794d by Serhiy Storchaka in branch 'default': Issue #23466: Fixed expected error message in test_format. https://hg.python.org/cpython/rev/11e6986c794d
msg240078 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-04 14:31
Good catch, Wolfgang! Definitely we should make test_format more unittest compatible.
History
Date User Action Args
2022-04-11 14:58:12 admin set github: 67654
2015-04-04 14:31:51 serhiy.storchaka set messages: +
2015-04-04 14:30:14 python-dev set messages: +
2015-04-04 14:07:05 wolma set nosy: + wolmamessages: +
2015-04-03 17:56:55 serhiy.storchaka set status: open -> closedresolution: fixedstage: patch review -> resolved
2015-04-03 17:54:11 python-dev set messages: +
2015-04-03 17:01:15 ethan.furman set messages: + stage: needs patch -> patch review
2015-04-03 09:10:56 serhiy.storchaka set messages: +
2015-04-03 08:48:40 vstinner set messages: +
2015-04-03 06:04:12 serhiy.storchaka set files: + bytes_format_overflow.patchmessages: +
2015-04-02 18:30:35 serhiy.storchaka set messages: +
2015-04-02 17:30:10 ethan.furman set status: closed -> openresolution: fixed -> (no value)messages: + stage: resolved -> needs patch
2015-04-02 15:20:00 ethan.furman set messages: +
2015-03-31 08:59:05 serhiy.storchaka set status: open -> closedassignee: serhiy.storchakaresolution: fixedstage: patch review -> resolved
2015-03-30 06:22:04 python-dev set nosy: + python-devmessages: +
2015-03-30 02:22:24 vstinner set messages: +
2015-03-30 01:51:52 ethan.furman set messages: +
2015-03-24 22:22:14 vstinner set nosy: + vstinnermessages: +
2015-03-24 16:37:23 serhiy.storchaka set messages: +
2015-02-16 21:50:29 serhiy.storchaka set files: + bytes_int_format.patchkeywords: + patchmessages: + stage: patch review
2015-02-15 20:01:30 vstinner set title: Inconsistency between str and bytes formatting of integers -> PEP 461: Inconsistency between str and bytes formatting of integers
2015-02-15 18:21:46 serhiy.storchaka create