Resampler sometimes adds extra bin with NaN · Issue #30353 · pandas-dev/pandas (original) (raw)

Code Sample

import pandas as pd idx = pd.timedelta_range(start='8H', freq='10S', periods=5040) s = pd.Series(range(len(idx)), index=idx) s.resample('3H').min()

Observed Output

08:00:00       0.0
11:00:00    1080.0
14:00:00    2160.0
17:00:00    3240.0
20:00:00    4320.0
23:00:00       NaN
Freq: 3H, dtype: float64

Expected Output

08:00:00       0.0
11:00:00    1080.0
14:00:00    2160.0
17:00:00    3240.0
20:00:00    4320.0
Freq: 3H, dtype: float64

Problem description

Downsampling and then applying (any) function to the Resampler object seems to add an extra bin in some cases. It is unclear when this exactly occurs, but these two conditions seem necessary (but not sufficient) to reproduce the bug:

• the resampling period does not exactly fit in the original index
• the original index starts with a non-zero Timedelta

This guess is based on the fact that changing '8H' to '0H' in the code above returns an expected output. Equally so does changing the resampling period to '2H'.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.5.final.0
python-bits : 64
OS : Linux
OS-release : 5.0.0-37-generic
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 0.25.3
numpy : 1.17.4
pytz : 2019.3
dateutil : 2.8.1
pip : 19.3.1
setuptools : 42.0.2.post20191203
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.10.3
IPython : 7.10.2
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : 3.1.1
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : 1.3.2
sqlalchemy : None
tables : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None