Issue 34752: warnings.warn fails silently with unicode input (original) (raw)

Created on 2018-09-20 14:15 by nparslow, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (5)
msg325880 - (view)	Author: Nicholas Parslow (nparslow)	Date: 2018-09-20 14:15
example: Python 2.7.14 |Anaconda, Inc.	(default, Dec 7 2017, 17:05:42) [GCC 7.2.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import warnings >>> warnings.warn(u'blé') >>> warnings.warn('blah') __main__:1: UserWarning: blah the warnings.warn_explicit function seems to be the part that is failing. On a machine which doesn't use anaconda it fails in an identical manner. if you do: >>> with warnings.catch_warnings(record=True) as w: ... warnings.warn(u'blé') ... >>> w [<warnings.WarningMessage object at 0x7ffadb35b190>] >>> w[0].message UserWarning(u'bl\xe9',) I can get that it fails with unicode (though it's annoying) but the silent fail is really bad imo.
msg326221 - (view)	Author: Karthikeyan Singaravelan (xtreak) *	Date: 2018-09-24 11:14
I think this a known limitation as per https://bugs.python.org/issue23637#msg239361 where non-ASCII characters would require byte string. I couldn't find the relevant documentation for it though. $ ./python.exe Python 2.7.15+ (remotes/upstream/2.7:69d0bc1430, Sep 24 2018, 16:02:09) [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import warnings >>> >>> warnings.warn(b'blé') __main__:1: UserWarning: blé >>> warnings.warn(u'blé') >>> Thanks
msg326222 - (view)	Author: Karthikeyan Singaravelan (xtreak) *	Date: 2018-09-24 11:30
I looked some more into the linked issue and I think this is an explicit decision and the comment in the code indicates that the warning will be lost : https://github.com/python/cpython/blob/69d0bc1430d2e9cddf0b39054ddcb86dbbe7927e/Lib/warnings.py#L34 . I understand your perspective on failing silently but I am proposing to close this since enabling the error by removing UnicodeDecodeError in the except clause will be an incompatible change at this point. Thanks
msg326227 - (view)	Author: Karthikeyan Singaravelan (xtreak) *	Date: 2018-09-24 12:38
Since Nick had some issue on adding a message to the issue I am adding the email sent to me as a message below : (sorry I can't see how to post a reply message on the bugs page, I'm still new to this) anyhow here's what I wanted to post: Thanks for finding the bug! Maybe a preferable behaviour would be to split the error types - an IOError could still pass (I'm really not sure what the ideal treatment in that case is - a Unicode error would still print a warning but it would be something like a generic 'warning: non-standard text found in warning text so only printing this' (I'm sure there's something better sounding / more consistent with other warnings but I don't know them that well) so the code would look something like: except (UnicodeError): message = "non-standard text found in warning text so only printing this" file.write(formatwarning(message, category, filename, lineno, line)) except (IOError): pass # the file (probably stderr) is invalid - this warning gets lost. but I'm not sure if an IOError inside the UnicodeError handling would still be caught (?) anyhow I'm not sure how standard/appropriate a generic warning would be within the broader context of python (?) I think for us we'll probably just bite the bullet and switch to python3 to avoid this kind of thing since we'll potentially have a lot of unicode messages in the near future. thanks again, Nick Parslow
msg384160 - (view)	Author: Zackery Spytz (ZackerySpytz) *	Date: 2021-01-01 11:01
Python 2.7 is no longer supported, so I think this issue should be closed.