BUG: Arithmetic, timezone and offsets operations affecting to NaT by sinhrks · Pull Request #6873 · pandas-dev/pandas (original) (raw)

NaT affected by some datetime related ops unexpectedly.

Arithmetic

Applying arithmetic ops to NaT is not handled properly. Based on numpy results, I understand that results should be all NaT as long as valid data is passed.

# current results
>>> pd.NaT + pd.offsets.Hour(1)
2262-04-11 01:12:43.145224192
>>> pd.NaT - pd.offsets.Hour(1))
OverflowError: Python int too large to convert to C long
>>> pd.NaT - pd.Timestamp('2011-01-01')
-734779 days, 0:00:00

# numpy
>>> np.datetime64('nat') + np.timedelta64(1, 'h')
NaT
>>> np.datetime64('nat') - np.timedelta64(1, 'h')
NaT
>>> np.datetime64('nat') - np.datetime64('2011-01-01')
NaT

Timezone

Closes #5546.

# current results
>>> idx = pd.DatetimeIndex(['2011-01-01 00:00', '2011-01-02 00:00', pd.NaT])
>>> idx.tz_localize('Asia/Tokyo')
<class 'pandas.tseries.index.DatetimeIndex'>
[2011-01-01 00:00:00+09:00, ..., 2262-04-10 00:12:43.145224192+09:00]
Length: 3, Freq: None, Timezone: Asia/Tokyo

>>> idx = pd.DatetimeIndex(['2011-01-01 00:00', '2011-01-02 00:00', pd.NaT], tz='US/Eastern')
>>> idx.tz_convert('Asia/Tokyo')
<class 'pandas.tseries.index.DatetimeIndex'>
[2011-01-01 14:00:00+09:00, ..., 2262-04-11 14:12:43.145224192+09:00]

Note I fixed DatetimeIndex, and I leave NatType still doesn't have tz_localize and tz_convert methods Timestamp has. Is it should be added?

Offsets

These have apply method which accepts Timestamp, but it cannot handle Nat.

# current result
>>> pd.offsets.Hour(1).apply(pd.NaT)
2262-04-11 01:12:43.145224192