BUG: nan-objects lookup fails in Python3.10 · Issue #41953 · pandas-dev/pandas (original) (raw)

I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
(optional) I have confirmed this bug exists on the master branch of pandas.

Due to this change in CPython: python/cpython@a07da09

the following assumption is no longer true:

// For PyObject_Hash holds:
// hash(0.0) == 0 == hash(-0.0)
// hash(X) == 0 if X is a NaN-value
// so it is OK to use it directly for doubles
Py_hash_t hash = PyObject_Hash(key);

Here is a failing example:

_____________________ TestIsin.test_different_nan_objects ______________________

self = <pandas.tests.test_algos.TestIsin object at 0x7f9ba1fe7250>

def test_different_nan_objects(self):
    # GH 22119
    comps = np.array(["nan", np.nan * 1j, float("nan")], dtype=object)
    vals = np.array([float("nan")], dtype=object)
    expected = np.array([False, False, True])
    result = algos.isin(comps, vals)

  tm.assert_numpy_array_equal(expected, result)

pandas/tests/test_algos.py:1039:

left = array([False, False, True]), right = array([False, False, False])