Always read metadata files as UTF-8 in setup.py by EliahKagan · Pull Request #1748 · gitpython-developers/GitPython (original) (raw)

Fixes #1747

This passes encoding="utf-8" to the open calls in setup.py, so the readme, version, and requirements files are always read as UTF-8, even on systems whose locale is not UTF-8. This fixes the bug described in #1747 where installation other than from a pre-built wheel would fail on many Windows systems using non-European languages.

The specific problem occurred with the README.md file. The requirements files are less likely to contain characters not in the ASCII subset, though maybe they could come to contain them, perhaps in comments. The VERSION file is even less likely to ever contain such characters. Nonetheless, for consistency, because it is a best practice, and because it appears to be the intent of the existing code, encoding="utf=8" is added for opening all of them.

This change is tested on a system whose locale uses Windows code page 936. Editable installation, as well as the other affected ways of installing (and building) described in #1747, are now working. (Installing from a pre-built wheel was never affected.)

I lean toward the position that it's not worth attempting a regression test for this, because although we do have test_installation, a truly reliable regression test for #1747 would require the presence of a locale in which the existing logic breaks, but in practice the systems where this happens are Windows, where a single locale (in the sense of "locale" relevant here) is systemwide. Furthermore, the fix is straightforward, following the current best practice of passing encoding="utf-8" explicitly where there is not reason to do otherwise. However, there may be a way to add a reasonably reliable test using mocking or other techniques, and I'd be pleased to pursue that if you consider it warranted.