BUG: pd.concat fails with GroupBy.head() and pd.StringDtype["pyarrow"] · Issue #51734 · pandas-dev/pandas (original) (raw)

Pandas version checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

df = pd.DataFrame({"A": pd.Series([], dtype=pd.StringDtype("pyarrow"))}) h = df.groupby("A")["A"].head(1) print(f"{h = }") res = pd.concat([h]) print(f"{res = }")

Issue Description

If the dataframe has dtype string[pyarrow], after running a groupby and taking a head of the series, the resulting Series seems to be correctly backed by pandas.core.arrays.string_arrow.ArrowStringArray, but can't be concatenated with pd.concat.

The snippet above produces an error:

h = Series([], Name: A, dtype: string)
Traceback (most recent call last):
  File "/Users/jbennet/Library/Application Support/JetBrains/PyCharm2022.3/scratches/arrow_concat.py", line 7, in <module>
    res = pd.concat([h])
  File "/Users/jbennet/mambaforge/envs/pandas-dev/lib/python3.8/site-packages/pandas/util/_decorators.py", line 331, in wrapper
    return func(*args, **kwargs)
  File "/Users/jbennet/mambaforge/envs/pandas-dev/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 381, in concat
    return op.get_result()
  File "/Users/jbennet/mambaforge/envs/pandas-dev/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 580, in get_result
    res = concat_compat(arrs, axis=0)
  File "/Users/jbennet/mambaforge/envs/pandas-dev/lib/python3.8/site-packages/pandas/core/dtypes/concat.py", line 133, in concat_compat
    return cls._concat_same_type(to_concat)
  File "/Users/jbennet/mambaforge/envs/pandas-dev/lib/python3.8/site-packages/pandas/core/arrays/arrow/array.py", line 802, in _concat_same_type
    arr = pa.chunked_array(chunks)
  File "pyarrow/table.pxi", line 1350, in pyarrow.lib.chunked_array
  File "pyarrow/error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: cannot construct ChunkedArray from empty vector and omitted type

Expected Behavior

I would expect to get an empty Series back.

Installed Versions

INSTALLED VERSIONS

commit : ce32601
python : 3.8.16.final.0
python-bits : 64
OS : Darwin
OS-release : 22.3.0
Version : Darwin Kernel Version 22.3.0: Thu Jan 5 20:50:36 PST 2023; root:xnu-8792.81.2~2/RELEASE_ARM64_T6020
machine : arm64
processor : arm
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 2.1.0.dev0+99.gce3260110f
numpy : 1.23.5
pytz : 2022.7.1
dateutil : 2.8.2
setuptools : 67.4.0
pip : 23.0.1
Cython : 0.29.33
pytest : 7.2.1
hypothesis : 6.68.2
sphinx : 4.5.0
blosc : None
feather : None
xlsxwriter : 3.0.8
lxml.etree : 4.9.2
html5lib : 1.1
pymysql : 1.0.2
psycopg2 : 2.9.3
jinja2 : 3.1.2
IPython : 8.11.0
pandas_datareader: None
bs4 : 4.11.2
bottleneck : 1.3.6
brotli :
fastparquet : 2023.2.0
fsspec : 2023.1.0
gcsfs : 2023.1.0
matplotlib : 3.6.3
numba : 0.56.4
numexpr : 2.8.3
odfpy : None
openpyxl : 3.1.0
pandas_gbq : None
pyarrow : 11.0.0
pyreadstat : 1.2.1
pyxlsb : 1.0.10
s3fs : 2023.1.0
scipy : 1.10.1
snappy :
sqlalchemy : 2.0.4
tables : 3.7.0
tabulate : 0.9.0
xarray : 2023.1.0
xlrd : 2.0.1
zstandard : 0.19.0
tzdata : None
qtpy : None
pyqt5 : None