Issue 32245: OSError: raw write() returned invalid length on latest Win 10 Consoles (original) (raw)

We need a test that reproduces this problem on a vanilla installation of Python 3.5. Include the system locale context as well, i.e. the ANSI codepage, OEM codepage, and active console output codepage. A reliable test will help determine whether this problem also affects legacy console I/O in Python 3.6. However, even if the problem affects 3.6, that doesn't mean there's anything we can reasonably do about it if the fault is the console host process (i.e. conhost.exe, running either ConhostV1.dll or the new ConhostV2.dll implementation).

A possible workaround in Python 3.5 would be to install and enable the win_unicode_console package. This package uses the console's native Unicode API instead of the legacy codepage API.

I was able to reproduce this problem in 3.6 in Windows 10 (1709), but only for code paths that call WriteFile, i.e. os.write and legacy standard I/O mode.

Writing to the console will block if there's an active text selection. The operation completes once the user copies the selection to the clipboard (e.g. by pressing enter). In the case of WriteFile, when the blocked call finally completes, the console mistakenly returns the number of internally written UTF-16 bytes.

This bug is not present with WriteConsoleA, which means there's a simple workaround. _Py_write could detect a console handle and call WriteConsoleA instead of _write.

The following test executes a write on a separate thread, after a timed delay that allows selecting text beforehand. Text can be selected via Ctrl+A, Shift+Up, or the mouse (the latter requires quick-edit mode or toggling mark mode).

import os
import sys
import ctypes
import msvcrt
import threading

kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
fd_stdout = sys.stdout.fileno()
h_stdout = msvcrt.get_osfhandle(fd_stdout)
n = ctypes.c_ulong()

def test_write():
    n.value = os.write(fd_stdout, b'spam\r\n')

def test_WriteFile():
    kernel32.WriteFile(h_stdout, b'spam\r\n', 6, ctypes.byref(n), None)

def test_WriteConsoleA():
    kernel32.WriteConsoleA(h_stdout, b'spam\r\n', 6, ctypes.byref(n), None)

For example, given manual text selection after starting the timer:

>>> threading.Timer(5, test_write).start()
>>> spam

>>> n
c_ulong(12)

>>> threading.Timer(5, test_WriteFile).start()
>>> spam

>>> n
c_ulong(12)

>>> threading.Timer(5, test_WriteConsoleA).start()
>>> spam

>>> n
c_ulong(6)

This test could be completely automated by combining SendInput (to set the session control-key state), GetConsoleWindow, and PostMessageW (WM_KEYDOWN, WM_KEYUP).