DatetimeIndex lookups tz inconsistency · Issue #18435 · pandas-dev/pandas (original) (raw)

TLDR: enforcing tzaware vs tznaive compat in DatetimeIndex comparisons (#18162) appears to be inconsistent with current slicing behavior.

The following example is based off of tests.series.test_indexing.test_getitem_setitem_datetimeindex:

dti = pd.date_range('1/1/1990', periods=50, freq='H', tz='US/Eastern')
ts = pd.Series(np.random.randn(50), index=dti)

lb = '1990-01-01 04:00:00'
rb = '1990-01-01 07:00:00'

The behavior that we want to enforce is #18162 requires that dti < pd.Timestamp(lb) should raise, as should dti < lb. At the moment they effectively get treated as UTC. But if we change this so that it does raise, the following from test_getitem_setitem_datetimeindex breaks pretty irrevocably:

ts[(ts.index >= lb) & (ts.index <= rb)]

There is also ts[lb:rb] which if we're being strict should raise, but at least we could make that still work. (BTW this implicitly casts lb and rb to US/Eastern, albeit in different places. So far that appears to be a related but distinct can of worms.)