BUG: Unexpected Behaviour pd.merge using left_on and right_index=True with single level MultiIndex · Issue #52331 · pandas-dev/pandas (original) (raw)

Pandas version checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

merge fails to join the DataFrames with a single level MultiIndex

expected successful join on first row =='A'

import pandas as pd left_col = ['A', 'B'] right_index = [('A',)] left_df = pd.DataFrame({"col":left_col}) right_df = pd.DataFrame(data={"b":[100]}, index=pd.MultiIndex.from_tuples(right_index, names=["col"])) pd.merge(left_df, right_df, left_on=["col"], right_index=True, how="left", validate="m:1")

merge succeeds to join the DataFrames if the MultiIndex has 2 levels

import pandas as pd left_col = ['A', 'B'] right_index = [('A','A',)] left_df = pd.DataFrame({"col1":left_col, "col2":left_col}) right_df = pd.DataFrame(data={"b":[100]}, index=pd.MultiIndex.from_tuples(right_index, names=["col1", "col2"])) pd.merge(left_df, right_df, left_on=["col1", "col2"], right_index=True, how="left", validate="m:1")

merge succeeds if it's an Index

import pandas as pd left_col = ['A', 'B'] right_index = ['A'] left_df = pd.DataFrame({"col":left_col}) right_df = pd.DataFrame(data={"b":[100]}, index=pd.Index(right_index, name="col")) pd.merge(left_df, right_df, left_on="col", right_index=True, how="left", validate="m:1")

Issue Description

In the first example, even though a list of 1 column is provided for left_on, matching the length of the index levels, merge fails to join successfully.
I think that given that it works for 2 levels, and that it works when there is a single level index, it should work for 1 as well, especially because it fails silently.
Alternatively, it should raise an Error if something is not correct.

Expected Behavior

First example returns:

pd.DataFrame.from_dict({'col': {0: 'A', 1: 'B'}, 'b': {0: None, 1: None}})

I would expect the first example to return the following DataFrame (same than example 3):

Installed Versions

INSTALLED VERSIONS

commit : 2e218d1
python : 3.10.7.final.0
python-bits : 64
OS : Darwin
OS-release : 22.1.0
Version : Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:30 PDT 2022; root:xnu-8792.41.9~2/RELEASE_ARM64_T8103
machine : arm64
processor : arm
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.UTF-8

pandas : 1.5.3
numpy : 1.24.2
pytz : 2023.3
dateutil : 2.8.2
setuptools : 63.2.0
pip : 23.0.1
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : None