pandas (original) (raw)

xref #33401 which discusses astype, #24559 which discusses constructors. I think these need to be considered jointly.

Two big-picture questions.

when we pass constructor(dt64values, tz=tz), do we interpret dt64values as UTC or wall-times?
when we pass constructor(dt64values).astype("M8[ns, tz]")`, do we interpret dt64values as UTC or wall-times?

Let's check the current behavior on master for Timestamp, DatetimeIndex, and Series (see code block below if you want to check my work)

cls	Timestamp	DatetimeIndex	Series
cls(dt64, dtype=dt64tz)	UTC	Wall	UTC
cls(dt64).astype(dt64tz)	N/A	Wall	UTC
cls(dt64tz).astype("M8[ns]")	N/A	UTC	UTC

If we were starting fresh, I would probably lean towards making these all interpret dt64 as wall-times. But because these are already mostly-UTC, it would be an easier change to bring the DatetimeIndex behavior in line with the others.

Thoughts? cc @jorisvandenbossche @mroeschke @jreback

values = (3600 * 10**9 * np.arange(4)).astype("datetime64[ns]")
dt64 = values[-1]

tz = "US/Pacific"
dtype = pd.DatetimeTZDtype(tz=tz)

ts = pd.Timestamp(dt64, tz=tz)
dti = pd.DatetimeIndex(values, dtype=dtype)
ser = pd.Series(values, dtype=dtype)

if_wall = pd.DatetimeIndex(values).tz_localize(tz)
if_utc = pd.DatetimeIndex(values).tz_localize("UTC").tz_convert(tz)

assert ts == if_utc[-1]
assert (dti == if_wall).all()
assert (ser == if_utc).all()

dti_naive = pd.DatetimeIndex(values)
ser_naive = pd.Series(values)

res_dti = dti_naive.astype(dtype)
res_ser = ser.astype(dtype)

assert (res_dti == dti).all()
assert (res_ser == ser).all()

# astype from tzaware -> tznaive
assert (dti.astype(dti_naive.dtype) == dti.tz_convert("UTC").tz_localize(None)).all()
assert (ser_naive.astype(ser.dtype) == ser).all()