BUG: eval fails for ExtensionArray · Issue #58748 · pandas-dev/pandas (original) (raw)

import pandas as pd

df = pd.DataFrame({'a': pd.array([1, 2, 3]), 'b': pd.array([4, 5, 6])})

this works as expected

print(df['a'] / df['b']) """ 0 0.25 1 0.4 2 0.5 dtype: Float64 """

this is not working

print(df.eval('a / b')) """ TypeError: Cannot interpret 'Int64Dtype()' as a data type """

This issue is coming from pint-pandas#137. ExtensionArray is used for this project and df.eval inlcuding division (\) fails with TypeError.

I was able to able to reproduce the pint-pandas issue with pandas build in IntegerArray as shown in the above example

Traceback (most recent call last): File "c:\CALC\DEV\pint pandas debug\test.py", line 17, in print(df.eval('a / b')) ^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\frame.py", line 4738, in eval return _eval(expr, inplace=inplace, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\eval.py", line 340, in eval parsed_expr = Expr(expr, engine=engine, parser=parser, env=env) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\expr.py", line 809, in init self.terms = self.parse() ^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\expr.py", line 828, in parse return self._visitor.visit(self.expr) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\expr.py", line 413, in visit return visitor(node, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\expr.py", line 419, in visit_Module return self.visit(expr, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\expr.py", line 413, in visit return visitor(node, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\expr.py", line 422, in visit_Expr return self.visit(node.value, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\expr.py", line 413, in visit return visitor(node, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\expr.py", line 535, in visit_BinOp return self._maybe_evaluate_binop(op, op_class, left, right) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\expr.py", line 502, in _maybe_evaluate_binop res = op(lhs, rhs) ^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\expr.py", line 538, in return lambda lhs, rhs: Div(lhs, rhs) ^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\ops.py", line 528, in init if not isnumeric(lhs.return_type) or not isnumeric(rhs.return_type): ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\laurent.mutricy\AppData\Local\miniconda3\Lib\site-packages\pandas\core\computation\ops.py", line 512, in isnumeric return issubclass(np.dtype(dtype).type, np.number) ^^^^^^^^^^^^^^^ TypeError: Cannot interpret 'Int64Dtype()' as a data type

Looking at the traceback it seams that pandas\core\computation\ops.isnumeric() does not handle properly dtypes from ExtensionArray class (or subclass). It was suggested to consider _is_numeric class attribute.

def isnumeric(dtype) -> bool: