Broken roundrip DatetimeIndex -> CategoricalIndex -> DatetimeIndex · Issue #18664 · pandas-dev/pandas (original) (raw)

Code Sample, a copy-pastable example if possible

In [2]: pd.DatetimeIndex(pd.CategoricalIndex(pd.DatetimeIndex(['2015-10-10'], tz='US/Eastern'))) Out[2]: DatetimeIndex(['2015-10-10 04:00:00'], dtype='datetime64[ns]', freq=None)

Problem description

Out[2] has no timezone information. Related to this comment

Notice that it makes sense to loose freq, as this is a property of a specific collection of dates, such as a DatetimeIndex is. Instead, tz is a property of each date, and it should be hence kept.

This is (I think) the reason why

In [2]: pd.core.dtypes.cast.maybe_cast_to_datetime(pd.Timestamp('2015-10-10'), None) Out[2]: array(['2015-10-10T00:00:00.000000000'], dtype='datetime64[ns]')

but

In [3]: pd.core.dtypes.cast.maybe_cast_to_datetime(pd.Timestamp('2015-10-10', tz='US/Eastern'), None)
Out[3]: DatetimeIndex(['2015-10-10 00:00:00-04:00'], dtype='datetime64[ns, US/Eastern]', freq=None)

Certainly related to #13783, #14052, #13238 and others, but probably requires a separate fix (as long as we don't have a real tz-aware dtype).

Expected Output

Out[2]: DatetimeIndex(['2015-10-10 00:00:00-04:00'], dtype='datetime64[ns, US/Eastern]', freq=None)

Output of pd.show_versions()

INSTALLED VERSIONS

commit: fdba133
python: 3.5.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-4-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8
LOCALE: it_IT.UTF-8

pandas: 0.22.0.dev0+301.gfdba13333
pytest: 3.2.3
pip: 9.0.1
setuptools: 36.7.0
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.19.0
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: 1.5.6
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: 1.2.0dev
tables: 3.3.0
numexpr: 2.6.1
feather: 0.3.1
matplotlib: 2.0.0
openpyxl: 2.3.0
xlrd: 1.0.0
xlwt: 1.3.0
xlsxwriter: 0.9.6
lxml: 4.1.1
bs4: 4.5.3
html5lib: 0.999999999
sqlalchemy: 1.0.15
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: 0.2.1