UnicodeDecodeError when setup.cfg contains non-ASCII and LC_ALL=C · Issue #1062 · pypa/setuptools (original) (raw)

cat >setup.py <<\EOF

-- coding: utf-8 --

from setuptools import setup setup( name='test', version='1.0', description='éàïñ', ) EOF

env LC_ALL=C python3 ./setup.py --description éàïñ cat >setup.cfg <<\EOF [metadata] name = test version = 1.0 description = éàïñ EOF cat >setup.py <<\EOF from setuptools import setup setup() EOF env LC_ALL=C python3 ./setup.py --description Traceback (most recent call last): File "./setup.py", line 2, in setup() File "/usr/lib/python3.6/distutils/core.py", line 121, in setup dist.parse_config_files() File "/usr/lib/python3.6/site-packages/setuptools/dist.py", line 355, in parse_config_files _Distribution.parse_config_files(self, filenames=filenames) File "/usr/lib/python3.6/distutils/dist.py", line 395, in parse_config_files parser.read(filename) File "/usr/lib/python3.6/configparser.py", line 697, in read self._read(fp, filename) File "/usr/lib/python3.6/configparser.py", line 1015, in _read for lineno, line in enumerate(fp, start=1): File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 75: ordinal not in range(128)

With PKG-INFO always written as UTF-8, I think it would make sense to load setup.cfg as UTF-8 too.

I see in the code that there's a setuptools/py36compat.py providing a patched parse_config_files but as seen in the traceback, it's not called. Is that normal?

According to the log it was added to fix #889, but adding a test for that fails:

diff --git i/setuptools/tests/test_config.py w/setuptools/tests/test_config.py index 8bd2a494..2b118b70 100644 --- i/setuptools/tests/test_config.py +++ w/setuptools/tests/test_config.py @@ -288,6 +288,15 @@ class TestMetadata: with get_dist(tmpdir) as dist: assert set(dist.metadata.classifiers) == expected

def test_no_interpolation(self, tmpdir):
```
   fake_env(
```
```
       tmpdir,
```
```
       '[metadata]\n'
```
```
       'description = %(message)s\n'
```
```
   )
```
```
   with get_dist(tmpdir) as dist:
```

       assert dist.metadata.description == '%(message)s'

class TestOptions:

Anyway, here is my attempt at fixing: master...benoit-pierre:fix_889_and_non-ascii_in_setup.cfg

I can make proper PR(s) for those changes that are OK.