pandas-dev/pandas (original) (raw)

Code Sample, a copy-pastable example if possible

Your code here

import pandas as pd import io

content = io.StringIO(''' time,val 212.23, 32 ''')

pd.set_option('compute.use_numexpr', False) print("use numexpr? %d" % pd.get_option('compute.use_numexpr'))

df = pd.read_csv( content, sep=',', usecols=['time', 'val'], dtype= {'val': 'Int64'}, # parse_dates=date_cols, ) q = "val == 32"

passing engine="python" fixes it

df_dest = df.query(q, )

throws with pandas 1.0

Traceback (most recent call last):
  File "test.py", line 21, in <module>
    df_dest = df.query(q, )
  File "/nix/store/0yyjswv58jk0fv1miyklmppvpd0p45sy-python3.7-pandas-1.0.1/lib/python3.7/sit
e-packages/pandas/core/frame.py", line 3231, in query
    res = self.eval(expr, **kwargs)
  File "/nix/store/0yyjswv58jk0fv1miyklmppvpd0p45sy-python3.7-pandas-1.0.1/lib/python3.7/sit
e-packages/pandas/core/frame.py", line 3346, in eval
    return _eval(expr, inplace=inplace, **kwargs)
  File "/nix/store/0yyjswv58jk0fv1miyklmppvpd0p45sy-python3.7-pandas-1.0.1/lib/python3.7/sit
e-packages/pandas/core/computation/eval.py", line 337, in eval
    ret = eng_inst.evaluate()
  File "/nix/store/0yyjswv58jk0fv1miyklmppvpd0p45sy-python3.7-pandas-1.0.1/lib/python3.7/sit
e-packages/pandas/core/computation/engines.py", line 73, in evaluate
    res = self._evaluate()
  File "/nix/store/0yyjswv58jk0fv1miyklmppvpd0p45sy-python3.7-pandas-1.0.1/lib/python3.7/sit
e-packages/pandas/core/computation/engines.py", line 114, in _evaluate
    return ne.evaluate(s, local_dict=scope)
  File "/nix/store/k6wr7w16q0aq9mfkf0w15wxlhx76sxqc-python3.7-numexpr-2.7.1/lib/python3.7/si
te-packages/numexpr/necompiler.py", line 822, in evaluate
    zip(names, arguments)]
  File "/nix/store/k6wr7w16q0aq9mfkf0w15wxlhx76sxqc-python3.7-numexpr-2.7.1/lib/python3.7/si
te-packages/numexpr/necompiler.py", line 821, in <listcomp>
    signature = [(name, getType(arg)) for (name, arg) in
  File "/nix/store/k6wr7w16q0aq9mfkf0w15wxlhx76sxqc-python3.7-numexpr-2.7.1/lib/python3.7/si
te-packages/numexpr/necompiler.py", line 703, in getType
    raise ValueError("unknown type %s" % a.dtype.name)
ValueError: unknown type object

Problem description

Dataframe.query fails when using UInt* dtypes see #25369 (comment).
My first idea to solve this was to disable numexpr but this isn't enough, one has to pass engine="python" to forcefully disable numexpr when calling query