BUG: empty dataframe indexing raises ValueError: buffer source array is read-only only when coming from db · Issue #57130 · pandas-dev/pandas (original) (raw)

Pandas version checks

Reproducible Example

import sqlite3

import pandas as pd

pd.options.mode.copy_on_write = True

con = sqlite3.connect(':memory:') df_sql = pd.read_sql( sql='SELECT 1 AS a, 2 AS b WHERE FALSE', con=con, index_col='a', )

df_plain = pd.DataFrame(columns=['b'], index=pd.Index([], name='a')) pd.testing.assert_frame_equal( left=df_sql, right=df_plain, check_column_type=True, check_exact=True, check_categorical=True, check_datetimelike_compat=True, )

df_plain.loc[df_plain.index > 69] # works df_sql.loc[df_sql.index > 69] # fails: ValueError: buffer source array is read-only

Issue Description

when reading an empty dataframe via read_sql (i.e. a query returning nothing) it is impossible to check for an index-conditions via .loc. It works fine with pd.options.mode.copy_on_write = False.

+ /tmp/pandas/venv/bin/ninja
[1/1] Generating write_version_file with a custom command
Traceback (most recent call last):
  File "/tmp/pandas/t.py", line 28, in <module>
    df_sql.loc[df_sql.index > 69] # fails: ValueError: buffer source array is read-only
               ^^^^^^^^^^^^^^^^^
  File "/tmp/pandas/pandas/core/ops/common.py", line 76, in new_method
    return method(self, other)
           ^^^^^^^^^^^^^^^^^^^
  File "/tmp/pandas/pandas/core/arraylike.py", line 56, in __gt__
    return self._cmp_method(other, operator.gt)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/pandas/pandas/core/indexes/base.py", line 7236, in _cmp_method
    result = ops.comp_method_OBJECT_ARRAY(op, self._values, other)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/pandas/pandas/core/ops/array_ops.py", line 129, in comp_method_OBJECT_ARRAY
    result = libops.scalar_compare(x.ravel(), y, op)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ops.pyx", line 32, in pandas._libs.ops.scalar_compare
  File "<stringsource>", line 663, in View.MemoryView.memoryview_cwrapper
  File "<stringsource>", line 353, in View.MemoryView.memoryview.__cinit__
ValueError: buffer source array is read-only

Expected Behavior

both, df_plain.loc[df_plain.index > 69] and df_sql.loc[df_sql.index > 69] should behave the same since the dfs are equal (assert_frame_equal).

Installed Versions

INSTALLED VERSIONS

commit : 56b5979
python : 3.11.7.final.0
python-bits : 64
OS : Linux
OS-release : 6.5.0-15-generic
Version : #15~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 12 18:54:30 UTC 2
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : de_DE.UTF-8
LOCALE : de_DE.UTF-8

pandas : 0+untagged.2.g56b5979
numpy : 1.26.3
pytz : 2023.4
dateutil : 2.8.2
setuptools : 69.0.3
pip : 23.3.2
Cython : 3.0.5
pytest : 8.0.0
hypothesis : 6.97.1
sphinx : 7.2.6
blosc : None
feather : None
xlsxwriter : 3.1.9
lxml.etree : 5.1.0
html5lib : 1.1
pymysql : 1.4.6
psycopg2 : 2.9.9
jinja2 : 3.1.3
IPython : 8.20.0
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.12.3
bottleneck : 1.3.7
dataframe-api-compat : None
fastparquet : 2023.10.1
fsspec : 2023.12.2
gcsfs : 2023.12.2post1
matplotlib : 3.8.2
numba : 0.58.1
numexpr : 2.9.0
odfpy : None
openpyxl : 3.1.2
pandas_gbq : None
pyarrow : 15.0.0
pyreadstat : 1.2.6
python-calamine : None
pyxlsb : 1.0.10
s3fs : 2023.12.2
scipy : 1.12.0
sqlalchemy : 2.0.25
tables : 3.9.2
tabulate : 0.9.0
xarray : 2024.1.1
xlrd : 2.0.1
zstandard : 0.22.0
tzdata : 2023.4
qtpy : None
pyqt5 : None