Exception when resampling ends on a DST change date · Issue #8653 · pandas-dev/pandas (original) (raw)

Hi all, I'm experiencing an exception when calling resample('D', how='mean') on a Series or Dataframe whose index (datetime-based) ends just on a DST change date.

As a workaround, I'm currently try/except-ing the ValueError and eventually adding a np.nan datapoint at the end of the series (effectively "walking over" the DST change date). At that point I call resample which completes correctly and filters away the np.nan datapoint anyway.

Snippet [1] reproduces the bug, calling resample on a short series which ends on Oct 26th, which was the DST change date for Europe/Rome. Raised exception is [2]. If relevant, [3] shows the installed versions info.

[1]

import datetime import pytz as tz import pandas as pd

rome = tz.timezone('Europe/Rome')

dr = [] for i in range(2): dp = datetime.datetime(2014, 10, 25) + datetime.timedelta(days=i) dr.append(rome.localize(dp))

series = {} for i, ddr in enumerate(dr): series[ddr] = i * 10

s1 = pd.Series(series) s1 = s1.resample('D', how='mean')

[2]

File "resample_this.py", line 18, in <module>
    s1 = s1.resample('D', how='mean')
  File "pandas/core/generic.py", line 2980, in resample
    return sampler.resample(self).__finalize__(self)
  File "pandas/tseries/resample.py", line 85, in resample
    rs = self._resample_timestamps()
  File "pandas/tseries/resample.py", line 314, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "pandas/core/groupby.py", line 2265, in aggregate
    return getattr(self, func_or_funcs)(*args, **kwargs)
  File "pandas/core/groupby.py", line 693, in mean
    return self._python_agg_general(f)
  File "pandas/core/groupby.py", line 1125, in _python_agg_general
    return self._wrap_aggregated_output(output)
  File "pandas/core/groupby.py", line 2332, in _wrap_aggregated_output
    return Series(output, index=index, name=name)
  File "pandas/core/series.py", line 212, in __init__
    data = SingleBlockManager(data, index, fastpath=True)
  File "pandas/core/internals.py", line 3326, in __init__
    ndim=1, fastpath=True)
  File "pandas/core/internals.py", line 2060, in make_block
    placement=placement)
  File "pandas/core/internals.py", line 75, in __init__
    len(self.values), len(self.mgr_locs)))
ValueError: Wrong number of items passed 2, placement implies 1

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Darwin
OS-release: 13.4.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.15.0
nose: None
Cython: None
numpy: 1.9.0
scipy: None
statsmodels: None
IPython: None
sphinx: 1.2.3
patsy: None
dateutil: 1.5
pytz: 2014.7
bottleneck: None
tables: None
numexpr: None
matplotlib: None
openpyxl: 2.1.1
xlrd: None
xlwt: 0.7.5
xlsxwriter: None
lxml: 3.4.0
bs4: 4.3.2
html5lib: None
httplib2: 0.9
apiclient: None
rpy2: None
sqlalchemy: None
pymysql: None
psycopg2: None