Issue 1037: Ill-coded identifier crashes python when coding spec is utf-8 (original) (raw)

Illegal identifier makes python crash on UTF-8 source codes/interpreters.

Python 3.0x (py3k:57555M, Aug 27 2007, 21:23:47) [GCC 3.4.6 [FreeBSD] 20060305] on freebsd6

compile(b'#coding:utf-8\n\xfc', '', 'exec') zsh: segmentation fault (core dumped) ./python

The problem is that tokenizer.c:verify_identifer doesn't check return value from PyUnicode_DecodeUTF8 but some invalid utf8 sequences could be there.