Support NDFrame.shift with EAs by TomAugspurger · Pull Request #22387 · pandas-dev/pandas (original) (raw)

It seems that IntervalArray may need some updates,

In [22]: idx = IntervalArray.from_breaks(pd.date_range('2017', periods=10))

In [23]: idx.shift()

TypeError Traceback (most recent call last) in () ----> 1 idx.shift()

~/sandbox/pandas/pandas/core/arrays/base.py in shift(self, periods) 422 return self.copy() 423 empty = self._from_sequence([self.dtype.na_value] * abs(periods), --> 424 dtype=self.dtype) 425 if periods > 0: 426 a = empty

~/sandbox/pandas/pandas/core/arrays/interval.py in _from_sequence(cls, scalars, dtype, copy) 193 @classmethod 194 def _from_sequence(cls, scalars, dtype=None, copy=False): --> 195 return cls(scalars, dtype=dtype, copy=copy) 196 197 @classmethod

~/sandbox/pandas/pandas/core/arrays/interval.py in new(cls, data, closed, dtype, copy, fastpath, verify_integrity) 138 139 return cls._simple_new(left, right, closed, copy=copy, dtype=dtype, --> 140 verify_integrity=verify_integrity) 141 142 @classmethod

~/sandbox/pandas/pandas/core/arrays/interval.py in _simple_new(cls, left, right, closed, copy, dtype, verify_integrity) 156 raise TypeError(msg.format(dtype=dtype)) 157 elif dtype.subtype is not None: --> 158 left = left.astype(dtype.subtype) 159 right = right.astype(dtype.subtype) 160

~/sandbox/pandas/pandas/core/indexes/numeric.py in astype(self, dtype, copy) 313 msg = ('Cannot convert Float64Index to dtype {dtype}; integer ' 314 'values are required for conversion').format(dtype=dtype) --> 315 raise TypeError(msg) 316 elif is_integer_dtype(dtype) and self.hasnans: 317 # GH 13149

TypeError: Cannot convert Float64Index to dtype datetime64[ns]; integer values are required for conversion

In [24]: idx = IntervalArray.from_breaks(range(10))

In [25]: idx.shift()

ValueError Traceback (most recent call last) in () ----> 1 idx.shift()

~/sandbox/pandas/pandas/core/indexes/numeric.py in astype(self, dtype, copy) 316 elif is_integer_dtype(dtype) and self.hasnans: 317 # GH 13149 --> 318 raise ValueError('Cannot convert NA to integer') 319 return super(Float64Index, self).astype(dtype, copy=copy) 320

ValueError: Cannot convert NA to integer

The basic issue is

    423         empty = self._from_sequence([self.dtype.na_value] * abs(periods),
--> 424                                     dtype=self.dtype)

The container (ndarray[self.dtype]) can't actually hold self.dtype.na_value (nan).

Does that change our opinion on this implementation?