| msg206741 - (view) |
Author: Konstantin Zemlyak (zart) |
Date: 2013-12-21 14:10 |
| Running `py.exe юникод.py` in cmd window fails: E:\>set PYLAUNCH_DEBUG=1 E:\>py юникод.py launcher build: 32bit launcher executable: Console File 'C:\Users\Zart\AppData\Local\py.ini' non-existent Using global configuration file 'C:\Windows\py.ini' Called with command line: .pymaybe_handle_shebang: read 211 bytes maybe_handle_shebang: BOM not found, using UTF-8 parse_shebang: found command: python3 locating Pythons in 64bit registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: "C:\Program Files\Python27\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python2\PCBuild\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locate_pythons_for_key: "C:\Program Files\Python2\PCBuild\amd64\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locate_pythons_for_key: "C:\Program Files\Python32\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locate_pythons_for_key: "C:\Program Files\Python33\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locating Pythons in native registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: "C:\Program Files (x86)\Python27\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python2\PCBuild\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locate_pythons_for_key: "C:\Program Files (x86)\Python2\PCBuild\amd64\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locate_pythons_for_key: "C:\Program Files (x86)\Python32\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locate_pythons_for_key: "C:\Program Files (x86)\Python33\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: ?????????????? ?????? ? ????? ?????, ????? ????? ??? ????? ????. found configured value 'python3=3.2-32' in environment search for Python version '3.2-32' found '"C:\Program Files (x86)\Python32\python.exe"' run_child: about to run '"C:\Program Files (x86)\Python32\python.exe" .py' C:\Program Files (x86)\Python32\python.exe: can't open file '.py': [Errno 2] No such file or directory child process exit code: 2 Note "Called with command line: .py" in output shows that filename was mangled very early on. Invoking `юникод.py` (which is associated with py.exe) directly works fine though. The problem lies in Windows handling of command-line arguments and fix to this is simple but non-obvious. Patch attached. |
|
|
| msg206743 - (view) |
Author: Konstantin Zemlyak (zart) |
Date: 2013-12-21 14:13 |
| Sorry, fixed whitespaces in the patch. |
|
|
| msg206744 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2013-12-21 14:19 |
| It looks like the wide character strings (wchar_t*) are misused. For example: error(RC_NO_PYTHON, L"Requested Python version (%s) ...", &p[1]); fwprintf(stdout, L"usage: %s ...\n\n", argv[0]); The %s formatter is for byte string (char*), "%ls" should be used instead. + _setmode(_fileno(stdout), _O_WTEXT); Extract of wprintf() documentation: "The wprintf() and vwprintf() functions perform wide-character output to stdout. stdout must not be byte oriented; see fwide(3) for more information." So _setmode() or fwide() should be used if I understood correctly. Or wprintf() should be replaced with printf() (still with "%ls" format)? wprintf("%ls") replaces unencodable character string arguments by ? (U+003F), whereas printf("%ls") and wprintf("%s") truncates the output at the first undecodable/unencodable character: http://unicodebook.readthedocs.org/en/latest/programming_languages.html#printf-functions-family So wprintf() is probably better here. |
|
|
| msg206745 - (view) |
Author: Konstantin Zemlyak (zart) |
Date: 2013-12-21 14:22 |
| I don't care much about debug output though it probably should be fixed. The point is that changing text mode of stdout has a weird side effect of fixing command-line arguments when invoking interactively from cmd.exe. |
|
|
| msg206760 - (view) |
Author: Konstantin Zemlyak (zart) |
Date: 2013-12-21 15:46 |
| There is something weird with my proposed fix. Right after submitting a bug with patch I've updated pythons on my system - 2.7.5 to 2.7.6, 3.3.2 to 3.3.3, and installed 3.4.0b1 - both 32- and 64-bit. Then my fixed py.exe stopped working. Then I've added _setmode for stdin/stdout/stderr and rebuilt both debug/release and x86/x64 versions: E:\>set PYLAUNCH_DEBUG=1 E:\>e:\cpython\PCbuild\py.exe юникод.py launcher build: 32bit launcher executable: Console launcher charset: Multi-byte File 'C:\Users\Zart\AppData\Local\py.ini' non-existent File 'e:\cpython\PCbuild\py.ini' non-existent Called with command line: .py maybe_handle_shebang: read 211 bytes maybe_handle_shebang: BOM not found, using UTF-8 parse_shebang: found command: python3 locating Pythons in 64bit registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: "C:\Program Files\Python27\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python2\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python2\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python32\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python33\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python34\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locating Pythons in native registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: "C:\Program Files (x86)\Python27\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python2\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python2\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python32\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python33\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python34\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. found configured value 'python3=3.3-32' in environment search for Python version '3.3-32' found '"C:\Program Files (x86)\Python33\python.exe"' run_child: about to run '"C:\Program Files (x86)\Python33\python.exe" .py' C:\Program Files (x86)\Python33\python.exe: can't open file '.py': [Errno 2] No such file or directory child process exit code: 2 E:\>e:\cpython\PCbuild\py_d.exe юникод.py launcher build: 32bit launcher executable: Console launcher charset: Multi-byte File 'C:\Users\Zart\AppData\Local\py.ini' non-existent File 'e:\cpython\PCbuild\py.ini' non-existent Called with command line: юникод.py maybe_handle_shebang: read 211 bytes maybe_handle_shebang: BOM not found, using UTF-8 parse_shebang: found command: python3 locating Pythons in 64bit registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: "C:\Program Files\Python27\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python2\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python2\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python32\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python33\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python34\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locating Pythons in native registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: "C:\Program Files (x86)\Python27\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python2\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python2\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python32\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python33\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python34\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. found configured value 'python3=3.3-32' in environment search for Python version '3.3-32' found '"C:\Program Files (x86)\Python33\python.exe"' run_child: about to run '"C:\Program Files (x86)\Python33\python.exe" юникод.py' Привет child process exit code: 0 E:\>e:\cpython\PCbuild\amd64\py.exe юникод.py launcher build: 64bit launcher executable: Console launcher charset: Multi-byte File 'C:\Users\Zart\AppData\Local\py.ini' non-existent File 'e:\cpython\PCbuild\amd64\py.ini' non-existent Called with command line: юникод.py maybe_handle_shebang: read 211 bytes maybe_handle_shebang: BOM not found, using UTF-8 parse_shebang: found command: python3 locating Pythons in 32bit registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: "C:\Program Files (x86)\Python27\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python2\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python2\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python32\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python33\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python34\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locating Pythons in native registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: "C:\Program Files\Python27\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python2\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python2\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python32\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python33\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python34\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. found configured value 'python3=3.3-32' in environment search for Python version '3.3-32' found '"C:\Program Files (x86)\Python33\python.exe"' run_child: about to run '"C:\Program Files (x86)\Python33\python.exe" юникод.py' Привет child process exit code: 0 E:\>e:\cpython\PCbuild\amd64\py_d.exe юникод.py launcher build: 64bit launcher executable: Console launcher charset: Multi-byte File 'C:\Users\Zart\AppData\Local\py.ini' non-existent File 'e:\cpython\PCbuild\amd64\py.ini' non-existent Called with command line: никод.py maybe_handle_shebang: read 211 bytes maybe_handle_shebang: BOM not found, using UTF-8 parse_shebang: found command: python3 locating Pythons in 32bit registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: "C:\Program Files (x86)\Python27\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python2\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python2\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python32\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python33\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python34\python.exe" is a 32bit executable locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files (x86)\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locating Pythons in native registry locate_pythons_for_key: unable to open PythonCore key in HKCU locate_pythons_for_key: "C:\Program Files\Python27\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python2\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python2\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python32\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python33\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python34\python.exe" is a 64bit executable locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. locate_pythons_for_key: "C:\Program Files\Python3\PCBuild\amd64\python.exe: Синтаксическая ошибка в имени файла, имени папки или метке тома. found configured value 'python3=3.3-32' in environment search for Python version '3.3-32' found '"C:\Program Files (x86)\Python33\python.exe"' run_child: about to run '"C:\Program Files (x86)\Python33\python.exe" никод.py' C:\Program Files (x86)\Python33\python.exe: can't open file '': [Errno 2] No such file or directory child process exit code: 2 Setting wide mode for stderr had fixed error messages in debug output. And looks like x64 debug build has off-by-one error and CRT behavior is wonky regarding command-line handling. So my patch doesn't really fix original problem yet exhibits some underlying crt bug. |
|
|
| msg206763 - (view) |
Author: Konstantin Zemlyak (zart) |
Date: 2013-12-21 16:38 |
| Some more fun stuff with command-line (I'm cutting output to few essential lines for easier reading): e:\cpython\PCbuild\py.exe юникод.py ... Called with command line: .py run_child: about to run '"C:\Program Files (x86)\Python33\python.exe" .py' C:\Program Files (x86)\Python33\python.exe: can't open file '.py': [Errno 2] No such file or directory child process exit code: 2 e:\cpython\PCbuild\py.exe e:\юникод.py ... Called with command line: e:\юникод.py run_child: about to run '"C:\Program Files (x86)\Python33\python.exe" e:\юникод.py' child process exit code: 0 E:\>e:\cpython\PCbuild\py.exe тест\unicode.py ... Called with command line: ест\unicode.py run_child: about to run '"C:\Program Files (x86)\Python33\python.exe" ест\unicode.py' C:\Program Files (x86)\Python33\python.exe: can't open file '': [Errno 2] No such file or directory child process exit code: 2 E:\>e:\cpython\PCbuild\py.exe e:\тест\unicode.py ... Called with command line: e:\тест\unicode.py run_child: about to run '"C:\Program Files (x86)\Python33\python.exe" e:\тест\unicode.py' child process exit code: 0 E:\>e:\cpython\PCbuild\py.exe "юникод.py" Called with command line: "юникод.py" run_child: about to run '"C:\Program Files (x86)\Python33\python.exe" "юникод.py"' child process exit code: 0 IOW, so long as command-line starts with ASCII character everything is fine. If not, then one or more characters gets mangled. Now I'm not sure whether it's a cmd.exe bug or C runtime one, and whether it's possible to workaround about it. |
|
|
| msg225162 - (view) |
Author: Mark Lawrence (BreamoreBoy) * |
Date: 2014-08-10 20:23 |
| Is this actually fixable? I only ask as we seem to have a whole lot of fun with anything involving cmd.exe as epitomized on #1602. |
|
|
| msg225496 - (view) |
Author: Steve Dower (steve.dower) * |
Date: 2014-08-18 16:02 |
| It should be fixable. In general, Unicode in the console is fine, but the CRT doesn't handle it well (as shown by the _setmode extension being able to fix it). The 'correct' fix for Unicode in the console is at http://www.siao2.com/2010/04/07/9989346.aspx and it basically comes down to "use the Windows API and not the CRT". It's certainly fixable here, though the general fix for Python itself is more difficult because we want/need to expose the bytes interface as well (that said, #1602 seems to have a good fix right now that just happens to be easily distributable as pure Python code, so there's little motivation to merge it in, especially since it will break back-compat). I don't know entirely whether _setmode is a correct fix here, or if the attached patch is sufficient, but it can be fixed. |
|
|
| msg225529 - (view) |
Author: Eryk Sun (eryksun) *  |
Date: 2014-08-19 11:12 |
| The problem is skip_whitespace mistakenly calls isspace instead of iswspace. http://hg.python.org/cpython/file/c0e311e010fc/PC/launcher.c#l48 isspace has undefined behavior when the argument is "not EOF or in the range of 0 through 0xFF": http://msdn.microsoft.com/en-us/library/y13z34da%28v=vs.100%29.aspx The display of debug messages should be handled in its own issue. IMO, setting stderr to _O_WTEXT mode or _O_U16TEXT mode looks reasonable. The launcher already uses wide-character strings and the wprintf family. It's just the default _O_TEXT mode ends up encoding to the console codepage. Regarding , %s in wide-character format strings is OK for VC++ 10: http://msdn.microsoft.com/en-us/library/hf4y5e3w%28v=vs.100%29.aspx This will be a legacy mode in VC++ 14 (CPython 3.5?): http://blogs.msdn.com/b/vcblog/archive/2014/06/18/crt-features-fixes-and-breaking-changes-in-visual-studio-14-ctp1.aspx P.S. In case someone wants an easy issue, I noticed a bug. There should only be one %s in the format string that starts on line 262: http://hg.python.org/cpython/file/c0e311e010fc/PC/launcher.c#l262 |
|
|
| msg241891 - (view) |
Author: Mark Lawrence (BreamoreBoy) * |
Date: 2015-04-23 21:34 |
| The bug reported in has been fixed, but there's another one a few lines up https://hg.python.org/cpython/file/bd656916586f/PC/launcher.c#l265 as there's only one % but two parameters. Although IIRC we'd get away with this the way C works, shouldn't we fix it anyway? |
|
|
| msg297792 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2017-07-06 01:26 |
| Is this still relevant or should it be closed? On Win10, I created a short script юникод.py using Save As from IDLE. py -2 юникод.py produces C:\Programs\Python27\python.exe: can't open file '??????.py': [Errno 22] Invalid argument If the patch fixes 2.7 and looks okay, maybe we should apply. If 'no' or 'no', maybe we should forget 2.7 running files with such names. On 3.5 and 3.6, the file runs without issue. The issue was opened with 3.3; 3.5 switched to a much more recent compiler, and I did not see any indication in the messages that this was tested on 3.5 before it was added. So perhaps for 3.5+, this is out-of-date. |
|
|
| msg297801 - (view) |
Author: Konstantin Zemlyak (zart) |
Date: 2017-07-06 04:37 |
| Terry J. Reedy wrote: > Is this still relevant or should it be closed? Should be closed. > On Win10, I created a short script юникод.py using Save As from IDLE. > > py -2 юникод.py produces > C:\Programs\Python27\python.exe: can't open file '??????.py': [Errno 22] Invalid argument > If the patch fixes 2.7 and looks okay, maybe we should apply. If 'no' or 'no', maybe we should forget 2.7 running files with such names. This outcome is expected and error comes from python itself, not from launcher. > On 3.5 and 3.6, the file runs without issue. The issue was opened with 3.3; 3.5 switched to a much more recent compiler, and I did not see any indication in the messages that this was tested on 3.5 before it was added. So perhaps for 3.5+, this is out-of-date. Launcher works fine now from my testing. |
|
|