BUG: tm.set_locale does not correctly set back the initial locale when locale.LC_ALL is used. tm.can_set_locale is impacted by the leak · Issue #46595 · pandas-dev/pandas (original) (raw)

Pandas version checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import locale import pandas._testing as tm

current_locale = locale.setlocale(locale.LC_ALL)

print(current_locale)

On my machine (FR):

>>> 'LC_COLLATE=C;LC_CTYPE=French_France.1252;LC_MONETARY=C;LC_NUMERIC=C;LC_TIME=C'

print(tm.can_set_locale("it_IT.utf8", locale.LC_ALL)) # False

new_current_locale = locale.setlocale(locale.LC_ALL) assert new_current_locale == current_locale # AssertionError: the contextmanager leaked print(new_current_locale) # 'it_IT.UTF-8' !!!!

Issue Description

The call to tm.can_set_locale actually contains a silent error when the inner tm.set_locale tries to set back the original locale. Because this error happens, the original locale is not set back !

The issue is actually due to a mistake in the code for set_locale: indeed the way it captures the initial locale (to set it back at the end) is wrong.

current_locale = locale.getlocale() is used, however this returns only the locale for the default category LC_CTYPE (see python doc)

What should be used instead is current_locale = locale.setlocale(lc_var). That way it is also working if lc_var is LC_ALL

Note that the example in the cpython documentation is misleading, I'll open an issue there as well.

Expected Behavior

No assertion error at the end of the example.

Installed Versions

AssertionError: C:\Miniconda3\envs\pandas-dev\lib\distutils\core.py