BUG: Timestamp.replace() behaves naively at DST boundaries (plus bonus segfault!) · Issue #7825 · pandas-dev/pandas (original) (raw)

In [1]: import pandas as pd

In [2]: t = pd.Timestamp('2013-11-3', tz='America/Chicago')

In [3]: t Out[3]: Timestamp('2013-11-03 00:00:00-0500', tz='America/Chicago')

In [4]: t.replace(hour=3) Out[4]: Timestamp('2013-11-03 03:00:00-0500', tz='America/Chicago')

This time doesn't exist.

In [5]: pd.Timestamp('2013-11-3 03:00:00', tz='America/Chicago') Out[5]: Timestamp('2013-11-03 03:00:00-0600', tz='America/Chicago')

In trying to replicate the definition of Timestamp.replace in tslib.pyx, I encountered this further issue:

In [6]: from datetime import datetime

In [7]: datetime.replace(t, hour=3) Segmentation fault: 11

Trying another tack, I found this:

In [1]: from datetime import datetime

In [2]: import pytz

In [3]: t = pytz.timezone('America/Chicago').localize(datetime(2013, 11, 3))

In [4]: t.strftime('%Y-%m-%d %H:%M:%S %Z%z') Out[4]: '2013-11-03 00:00:00 CDT-0500'

In [5]: t.replace(hour=3).strftime('%Y-%m-%d %H:%M:%S %Z%z') Out[5]: '2013-11-03 03:00:00 CDT-0500'

This time still never happened

In summary: datetime is horribly broken, and the current workaround doesn't actually get all the way around. I know breaking backward compatibility is painful, and throwing the stdlib out the window is even more so, but this situation reminds me of nothing so forcefully as Microsoft Excel's 1900/1904 issue.

Version information:

In [1]: from pandas.util.print_versions import show_versions

In [2]: show_versions()

INSTALLED VERSIONS

commit: None python: 2.7.5.final.0 python-bits: 64 OS: Darwin OS-release: 13.3.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8

pandas: 0.14.1 nose: 1.3.0 Cython: None numpy: 1.8.1 scipy: 0.12.1 statsmodels: None IPython: 1.1.0 sphinx: None patsy: None scikits.timeseries: None dateutil: 2.2 pytz: 2014.4 bottleneck: None tables: None numexpr: None matplotlib: 1.3.0 openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml: None bs4: None html5lib: None httplib2: None apiclient: None rpy2: None sqlalchemy: None pymysql: None psycopg2: None