Issue 28385: Bytes objects should reject all formatting codes with an error message (original) (raw)

Created on 2016-10-07 14:28 by socketpair, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
issue28385.diff berker.peksag,2016-10-08 12:08 review
object-__format__.patch serhiy.storchaka,2016-10-08 14:06 review
object-__format__-2.patch serhiy.storchaka,2016-10-18 07:42 review
Messages (20)
msg278244 - (view) Author: Марк Коренберг (socketpair) * Date: 2016-10-07 14:28
$ python3 -c "'{0:s}'.format(b'qwe')" Traceback (most recent call last): File "", line 1, in TypeError: non-empty format string passed to object.__format__ Spent many hours to detect bug in my code.
msg278249 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-10-07 15:44
Bytes don't support formatting codes, so they are passed through to object, which doesn't support any formatting codes, and produces the error message you see. Since all other built in types reject invalid codes with a message that mentions their type, I think bytes should too. This would change the message to: ValueError: Unknown format code 's' for object of type 'bytes' Would that have lessened your confusion sufficiently?
msg278250 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-10-07 15:56
Yes, I think we should implement this for object.__format__. Marking as easy.
msg278260 - (view) Author: Марк Коренберг (socketpair) * Date: 2016-10-07 18:54
Yes, that message will be sufficient. Perfectly.
msg278293 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-10-08 12:08
Here's a patch. Changes in test_builtin might be redundant, but I added them anyway.
msg278301 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-10-08 13:48
I left a comment on Rietveld, but repeating it here because for me those messages often go to spam: I think you want to make this more general, by modifying the message for the TypeError that follows. Any type that doesn't provide it's own __format__ should produce an error when using a non-empty format string. For example: >>> format({}, 'a') Traceback (most recent call last): File "", line 1, in TypeError: non-empty format string passed to object.__format__ Would be better as: TypeError: non-empty format string passed to __format__ for object of type "class <'dict'>" (Or some prettier way to print out the type).
msg278302 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-08 14:06
Sorry, but it looks to me that .diff solves wrong issue. The issue is not in bytes, but in default __format__. object.__format__ should raise an error message that contains the name of the actual type. There are other issues with object.__format__, following patch tries to solve them. 1. Error message now contains the name of the actual type. 2. Format spec is checked before converting the value to str. Converting bytes and bytearray to str can raise a BytesWarning, and it is better if the type of this error doesn't depend on the -b option. 3. Since format spec is always empty, avoid calling PyObject_Format(). In theory PyObject_Str() can raise a subclass of str with non-trivial __format__, but I think we can ignore this subtle behavior difference.
msg278831 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-10-18 00:30
LGTM, although I'm not so sure about your #3. Maybe it should be a separate issue and raised on python-dev? But I don't feel strongly enough about it to argue the point. Thanks!
msg278845 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-18 07:42
Removed item 3 from updated patch.
msg278846 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-18 07:53
From : > This is basically the definition of object.__format__: > > def __format__(self, specifier): > if len(specifier) == 0: > return str(self) > raise TypeError('non-empty format string passed to object.__format__') This simple algorithm is implemented in object-__format__.patch. But current implementation uses ``format(str(self), '')`` instead of just ``str(self)``. object-__format__-2.patch keeps this.
msg279736 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-10-30 17:38
New changeset 92cae79fa5d9 by Serhiy Storchaka in branch '3.5': Issue #28385: An error message when non-empty format spec is passed to https://hg.python.org/cpython/rev/92cae79fa5d9 New changeset 0a985f7c6731 by Serhiy Storchaka in branch '3.6': Issue #28385: An error message when non-empty format spec is passed to https://hg.python.org/cpython/rev/0a985f7c6731 New changeset 6e8183abcc35 by Serhiy Storchaka in branch 'default': Issue #28385: An error message when non-empty format spec is passed to https://hg.python.org/cpython/rev/6e8183abcc35
msg279739 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-10-30 18:54
Buildbots are failing: ====================================================================== FAIL: test_errors (test.test_fstring.TestCase) (str="f'{(lambda: 0):x}'") ---------------------------------------------------------------------- TypeError: unsupported format string passed to function.__format__ During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/buildarea/3.6.angelico-debian-amd64/build/Lib/test/test_fstring.py", line 20, in assertAllRaise eval(str) AssertionError: "non-empty" does not match "unsupported format string passed to function.__format__" ====================================================================== FAIL: test_errors (test.test_fstring.TestCase) (str="f'{(0,):x}'") ---------------------------------------------------------------------- TypeError: unsupported format string passed to tuple.__format__ During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/buildarea/3.6.angelico-debian-amd64/build/Lib/test/test_fstring.py", line 20, in assertAllRaise eval(str) AssertionError: "non-empty" does not match "unsupported format string passed to tuple.__format__" ---------------------------------------------------------------------- Ran 48 tests in 1.389s
msg279742 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-30 19:13
What is better? 1. Replace "unsupported format string passed to" back to "non-empty format string passed to " in generated error message? Note that full error message is changed in any case, this is the purpose of this issue. 2. Change the pattern "non-empty" to "unsupported" in f-string tests. Or to other part of new error message. It may be changed again in future.
msg279756 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-10-31 01:03
I would change the f-string tests. I realize it's a fragile test, but it's the only way to test the strings in the exception.
msg279762 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-10-31 06:13
New changeset f43078ec598c by Serhiy Storchaka in branch '3.6': Update the f-string test broken in issue #28385. https://hg.python.org/cpython/rev/f43078ec598c New changeset 931410a04240 by Serhiy Storchaka in branch 'default': Update the f-string test broken in issue #28385. https://hg.python.org/cpython/rev/931410a04240
msg279763 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-31 06:19
Shouldn't the type of error be changed from TypeError to ValueError?
msg279786 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-10-31 12:47
TypeError is documented as "Raised when an operation or function is applied to an object of inappropriate type". I think that fits this case: you're applying some format code to a type that doesn't support it.
msg279787 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-31 12:56
But on other hand, the error depends on the value of format specifier: >>> format('abc', '') 'abc' >>> format('abc', 'j') Traceback (most recent call last): File "", line 1, in ValueError: Unknown format code 'j' for object of type 'str' >>> format([1, 2, 3], '') '[1, 2, 3]' >>> format([1, 2, 3], 'j') Traceback (most recent call last): File "", line 1, in TypeError: unsupported format string passed to list.__format__ If keep TypeError I think it would be better to restore former error message "non-empty format string".
msg279844 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-11-01 00:29
I don't have a strong feeling one way or the other. I'd be surprised if anyone is catching these errors.
msg279986 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-03 11:36
The proposition of making default __format__ equivalent to str() will be addressed in separate issue. Python-Dev thread: https://mail.python.org/pipermail/python-dev/2016-October/146765.html.
History
Date User Action Args
2022-04-11 14:58:38 admin set github: 72571
2016-11-03 11:36:21 serhiy.storchaka set status: open -> closedresolution: fixedmessages: + stage: patch review -> resolved
2016-11-01 00:29:48 eric.smith set messages: +
2016-10-31 12:56:51 serhiy.storchaka set messages: +
2016-10-31 12:47:06 eric.smith set messages: +
2016-10-31 06:19:44 serhiy.storchaka set messages: +
2016-10-31 06:13:53 python-dev set messages: +
2016-10-31 01:03:27 eric.smith set messages: +
2016-10-30 19:13:25 serhiy.storchaka set messages: +
2016-10-30 18:54:55 ned.deily set nosy: + ned.deilymessages: +
2016-10-30 17:38:46 python-dev set nosy: + python-devmessages: +
2016-10-30 17:26:16 serhiy.storchaka set assignee: serhiy.storchaka
2016-10-18 07:53:46 serhiy.storchaka set messages: +
2016-10-18 07:44:54 vstinner set nosy: + vstinner
2016-10-18 07:42:39 serhiy.storchaka set files: + object-__format__-2.patchmessages: +
2016-10-18 00:30:46 eric.smith set messages: +
2016-10-08 14:06:52 serhiy.storchaka set files: + object-__format__.patchnosy: + serhiy.storchakamessages: +
2016-10-08 13:48:52 eric.smith set messages: +
2016-10-08 12:08:27 berker.peksag set files: + issue28385.diffnosy: + berker.peksagmessages: + keywords: + patchstage: needs patch -> patch review
2016-10-07 18:54:33 socketpair set messages: +
2016-10-07 15:56:19 eric.smith set keywords: + easy (C)messages: + components: + Interpreter Core, - Library (Lib)
2016-10-07 15:44:20 r.david.murray set nosy: + eric.smith, r.david.murraytitle: Non-informative exception while formatting strings. -> Bytes objects should reject all formatting codes with an error messagemessages: + type: behaviorstage: needs patch
2016-10-07 14:28:40 socketpair create