ENH: Modify .any() to return Null if all values are null · Issue #41967 · pandas-dev/pandas (original) (raw)

Is your feature request related to a problem?

For sparce data it is particuarly helpful to preserve null values when doing reduction operations. I end up using this (see below) wrapper function to achieve the intended result.

def any_nan(x):
    if pd.isnull(x).all():
        return pd.NA
    else:
        return x.any(skipna=True)

output = df.column_to_operate_on.apply(any_nan) 

Describe the solution you'd like

The ask is to add a configuration parameter to. .any() and .all() that optionally preserves null values when returning the array. Skipna doesn't seem to have to correct functionality as it will return false, if all the values of the array are null. Ideally, with the toggleable parameter, .any() would return np.nan/pd.NA if all values in the array are null.

API breaking implications

This shouldn't change the api. It will just add an optional parameter

Describe alternatives you've considered

pd.NA