BUG: df[frozenset] fails if (other) column names are not unique · Issue #41062 · pandas-dev/pandas (original) (raw)

I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
(optional) I have confirmed this bug exists on the master branch of pandas.

Code Sample, a copy-pastable example

import pandas as pd

k = frozenset(["KEY"])

df1 = pd.DataFrame([[1,2,3,4], [5,6,7,8]], columns=[k, "B", "C", "D"]) df2 = pd.DataFrame([[1,2,3,4], [5,6,7,8]], columns=[k, "B", "C", "C"])

print(df1[k]) # works print(df2[k]) # KeyError: "None of [Index(['KEY'], dtype='object')] are in the [columns]"

Problem description

Indexing a dataframe with a frozenset raises a KeyError if column names are not unique (the name of the indexed column is unique):

Traceback (most recent call last): File "", line 1, in File "C:\test\venv124\lib\site-packages\pandas\core\frame.py", line 3030, in getitem indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1] File "C:\test\venv124\lib\site-packages\pandas\core\indexing.py", line 1266, in _get_listlike_indexer self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing) File "C:\test\venv124\lib\site-packages\pandas\core\indexing.py", line 1308, in _validate_read_indexer raise KeyError(f"None of [{key}] are in the [{axis_name}]") KeyError: "None of [Index(['KEY'], dtype='object')] are in the [columns]"

Indexing another dataframe that only differs by having unique column names works, so using a frozenset as column name seems to be ok.

The traceback seems to indicate that a "listlike_indexer" is involved - maybe with non-unique column names a different indexer is used that has problems with non-atomic column names, turning the column name frozenset(["KEY"]) into "KEY". Just a guess based on the error message, I don't know anything about how pandas indexers work.

Expected Output

Indexing works with a column name of arbitrary type regardless of whether the other column names are unique or not.

Output of `pd.show_versions()`

INSTALLED VERSIONS

commit : 2cb9652
python : 3.9.4.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.18362
machine : AMD64
...