Issue 37106: python re.escape doesn't escape some special characters. (original) (raw)

Issue37106

Created on 2019-05-31 04:37 by MANI M, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (8)
msg344020 - (view)	Author: MANI M (MANI M)	Date: 2019-05-31 04:37
Recently I figured out an issue in python3 re which doesn't escape some special characters. Not sure whether this bug has been reported already. Have attached screenshots for your reference. Steps to reproduce: 1. wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz 2. tar -xvzf Python-3.7.3.tar.xz 3. cd Python-3.7.3 4. ./configure 5. make 6. make install. GCC version: gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36) OS: CentOS Linux release 7.6.1810 (Core)
msg344022 - (view)	Author: Karthikeyan Singaravelan (xtreak) *	Date: 2019-05-31 04:46
Please consider posting text content instead of images for better accessibility. This could be due to . ➜ cpython git:(master) python3.6 Python 3.6.4 (default, Mar 12 2018, 13:42:53) [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import re >>> a = r"Hello'`~world" >>> re.escape(a) "Hello\\'\\`\\~world" ➜ cpython git:(master) python3.7 Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import re >>> a = r"Hello'`~world" >>> re.escape(a) "Hello'`\\~world"
msg344026 - (view)	Author: MANI M (MANI M)	Date: 2019-05-31 05:37
Thanks a lot for the info. May I know in what version of python the patches are applied? Because still 3.7.3 seems to have the issue.
msg344027 - (view)	Author: Karthikeyan Singaravelan (xtreak) *	Date: 2019-05-31 06:04
It's a behavior change from 3.6 and it's present from 3.7.0a1
msg344032 - (view)	Author: Eric V. Smith (eric.smith) *	Date: 2019-05-31 07:40
Could you show a problem caused by the characters that are unescaped? I assume you're talking about the ` and ' characters, since that's what your example shows. But those aren't listed as "special characters" (https://docs.python.org/3.5/library/re.html#regular-expression-syntax), so I'm not sure what problem would be caused by them being unescaped.
msg344038 - (view)	Author: MANI M (MANI M)	Date: 2019-05-31 08:51
I've scripts which insert data into MySQL database. The values may contain symbols. Hence in order to escape that I use re.escape(). @erik.smith isn't re.escape() supposed to escape all the symbols. If not why is this introduced in 3.7 whereas previous versions behave differently. Example snippet: import pymysql from re import escape def db_connection(): ...... ...... ...... # This throws error. query = " insert into table(column) values('{}'.format(escape("Hello'`~world")))
msg344040 - (view)	Author: MANI M (MANI M)	Date: 2019-05-31 08:54
sorry my bad query = "insert into table(column) values('{}')".format(escape("Hello'`~world"))
msg344041 - (view)	Author: Eric V. Smith (eric.smith) *	Date: 2019-05-31 09:02
re.escape() is designed to only escape characters that have special meaning in regular expressions. It is not a general purpose escaping mechanism, and it is especially dangerous to use it for building SQL statements. You should be using parameterized SQL queries. See https://en.wikipedia.org/wiki/SQL_injection and for example https://stackoverflow.com/questions/1633332/how-to-put-parameterized-sql-query-into-variable-and-then-execute-in-python In any event, it seems that re.escape() is working as designed, so I'm going to close this.