BUG: Agg and Apply with custom function improperly ignore column labels when function returns a dict-like · Issue #37544 · pandas-dev/pandas (original) (raw)

Hello folks! I've discovered an odd corner case regression that I hope can be fixed.

The problem is that if you are running an aggregation/apply on a dataframe using a custom function - if that function returns a dict-like object, it doesn't use the appropriate column labels in the original dataframe.

As a simple replication

df = pd.DataFrame([[1, 2, 3],
                   [4, 5, 6],
                   [7, 8, 9],
                   [np.nan, np.nan, np.nan]],
                  columns=['A', 'B', 'C'])

print(df.agg(lambda x : {"sum" :np.sum(x)}))

prints

0    {'sum': 12.0}
1    {'sum': 15.0}
2    {'sum': 18.0}

instead of the expected

A    {'sum': 12.0}
B    {'sum': 15.0}
C    {'sum': 18.0}

This is a regression/breaking change since in prior versions of pandas (I believe 1.0.X and earlier) the expected results were produced.