BUG: Incompatible tolerance error in merge_asof with comaptible types. · Issue #35558 · pandas-dev/pandas (original) (raw)

[X ] I have checked that this issue has not already been reported.
[X ] I have confirmed this bug exists on the latest version of pandas.
Crashes in 1.1.0, but works in 1.0.5
[X ] (optional) I have confirmed this bug exists on the master branch of pandas.

Code Sample, a copy-pastable example

import pandas as pd

dr1 = pd.date_range(start='1/1/2020', end='2/1/2020') dr2 = pd.date_range(start='1/1/2020', end='1/20/2020', freq='2D') + pd.Timedelta(seconds=0.4)

df1 = pd.DataFrame({'val1': 'foo'}, index=pd.DatetimeIndex(dr1)) df2 = pd.DataFrame({'val2': 'bar'}, index=pd.DatetimeIndex(dr2))

td = pd.Timedelta(seconds=0.5) pd.merge_asof(df2, df1, left_index=True, right_index=True, tolerance=td)

Problem description

When trying to use merge_asof in 1.1.0 (and in master), I get message about "incompatible tolerance" even though I'm using pandas.DatetimeIndex and pandas.Timedelta for the tolerance.

Well, technically, I actually get UnbondLocalError because a local variable is referenced before assignment while trying to generate the error message, but it looks like the underlying problem is something to do with an incompatible tolerance error.

Here's the traceback:

UnboundLocalError Traceback (most recent call last) in 1 td = pd.Timedelta(seconds=0.5) ----> 2 pd.merge_asof(df2, df1, left_index=True, right_index=True, tolerance=td)

~/miniconda3/envs/pandas_problem/lib/python3.8/site-packages/pandas/core/reshape/merge.py in merge_asof(left, right, on, left_on, right_on, left_index, right_index, by, left_by, right_by, suffixes, tolerance, allow_exact_matches, direction) 544 4 2016-05-25 13:30:00.048 AAPL 98.00 100 NaN NaN 545 """ --> 546 op = _AsOfMerge( 547 left, 548 right,

~/miniconda3/envs/pandas_problem/lib/python3.8/site-packages/pandas/core/reshape/merge.py in init(self, left, right, on, left_on, right_on, left_index, right_index, by, left_by, right_by, axis, suffixes, copy, fill_method, how, tolerance, allow_exact_matches, direction) 1571 self.direction = direction 1572 -> 1573 _OrderedMerge.init( 1574 self, 1575 left,

~/miniconda3/envs/pandas_problem/lib/python3.8/site-packages/pandas/core/reshape/merge.py in init(self, left, right, on, left_on, right_on, left_index, right_index, axis, suffixes, copy, fill_method, how) 1465 1466 self.fill_method = fill_method -> 1467 _MergeOperation.init( 1468 self, 1469 left,

~/miniconda3/envs/pandas_problem/lib/python3.8/site-packages/pandas/core/reshape/merge.py in init(self, left, right, how, on, left_on, right_on, axis, left_index, right_index, sort, suffixes, copy, indicator, validate) 650 self.right_join_keys, 651 self.join_names, --> 652 ) = self._get_merge_keys() 653 654 # validate the merge keys dtypes. We may need to coerce

~/miniconda3/envs/pandas_problem/lib/python3.8/site-packages/pandas/core/reshape/merge.py in _get_merge_keys(self) 1667 1668 msg = ( -> 1669 f"incompatible tolerance {self.tolerance}, must be compat " 1670 f"with type {repr(lk.dtype)}" 1671 )

UnboundLocalError: local variable 'lk' referenced before assignment

Expected Output

Here's what I get in version 1.0.4, and what I think I should get.

                         val2 val1
2020-01-01 00:00:00.400  bar  foo
2020-01-03 00:00:00.400  bar  foo
2020-01-05 00:00:00.400  bar  foo
2020-01-07 00:00:00.400  bar  foo
2020-01-09 00:00:00.400  bar  foo
2020-01-11 00:00:00.400  bar  foo
2020-01-13 00:00:00.400  bar  foo
2020-01-15 00:00:00.400  bar  foo
2020-01-17 00:00:00.400  bar  foo
2020-01-19 00:00:00.400  bar  foo

Output of `pd.show_versions()`

INSTALLED VERSIONS

commit : a4203cf
python : 3.8.5.final.0
python-bits : 64
OS : Linux
OS-release : 5.4.0-7634-generic
Version : #38~~1595345317~~20.04~a8480ad-Ubuntu SMP Wed Jul 22 15:13:45 UTC
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.0.dev0+29.ga4203cf8d
numpy : 1.19.1
pytz : 2020.1
dateutil : 2.8.1
pip : 20.1.1
setuptools : 49.2.0.post20200714
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.16.1
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None