Message 171938 - Python tracker (original) (raw)

note, I was asked to report this issue, posted on the py dev mailing list: see - http://code.activestate.com/lists/python-dev/118015/

We've run into an issue recently with blender3d on ms-windows where we want to enforce the encoding is UTF-8 with the embedded python interpreter. (the encoding defaults to cp437).

I naively thought setting the environment variable before calling Py_Initialize() would work, but the way python DLL loads, it gets its own environment variables that cant be modified directly [1]. eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape");

We had bug reports by windows users not able to export files because the stdout errors on printing paths with unsupported encoding. [2],[3]

Of course we could distribute blender with a bat file launcher that sets env variables, or ask the user to set their env variable - but I dont think this is really a good option.

I tried overriding the stderr & stdout, but this caused another bug on exiting, giving an assert in MSVCR90.DLL's write.c (called from python32_d.dll): _VALIDATE_CLEAR_OSSERR_RETURN((_osfile(fh) & FOPEN), EBADF, -1);

import sys, io sys.stdout = sys.stdout = io.TextIOWrapper(io.open(sys.stdout.fileno(), "wb", -1), encoding='utf-8', errors='surrogateescape', newline="\n", line_buffering=True) sys.stderr = sys.stderr = io.TextIOWrapper(io.open(sys.stderr.fileno(), "wb", -1), encoding='utf-8', errors='surrogateescape', newline="\n", line_buffering=True)

IMHO either of these solutions would be fine.

• have a PyOS_PutEnv() function, gettext has gettext_putenv() to workaround this problem.

• manage this the same as Py_GetPythonHome(), which can be defined by the embedding application to override the default.

[1] http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe [2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750 [3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555