Read from HDF with empty where throws an error (original) (raw)

Code Sample

df = pd.DataFrame(np.random.rand(4,4))

where = '' with pd.HDFStore('test.h5') as store: store.put('df', df, 't') store.select('df', where = where)

Problem description

Wanted to be able construct "by hands" and save where condition for later, so declare it as variable. But some times constructed where becomes empty and code throws an error.

Traceback (most recent call last):

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3267, in run_code exec(code_obj, self.user_global_ns, self.user_ns)

File "", line 6, in store.select('df', where = where)

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/io/pytables.py", line 740, in select return it.get_result()

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/io/pytables.py", line 1518, in get_result results = self.func(self.start, self.stop, where)

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/io/pytables.py", line 733, in func columns=columns)

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/io/pytables.py", line 4254, in read if not self.read_axes(where=where, **kwargs):

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/io/pytables.py", line 3443, in read_axes self.selection = Selection(self, where=where, **kwargs)

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/io/pytables.py", line 4815, in init self.terms = self.generate(where)

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/io/pytables.py", line 4828, in generate return Expr(where, queryables=q, encoding=self.table.encoding)

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/core/computation/pytables.py", line 548, in init self.terms = self.parse()

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/core/computation/expr.py", line 766, in parse return self._visitor.visit(self.expr)

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/core/computation/expr.py", line 331, in visit return visitor(node, **kwargs)

File "/home/beforeflight/Coding/Python/venvs/main/lib/python3.7/site-packages/pandas/core/computation/expr.py", line 335, in visit_Module raise SyntaxError('only a single expression is allowed')

File "", line unknown SyntaxError: only a single expression is allowed

Expected Output

When empty string is passed to where - just select whole DataFrame. It may be easily achieved by changing last statement to store.select('df', where = where if where else None). But it would be better to add this checking inside pandas, so user may not worry about it all the times using selection from HDF with where.

Output of pd.show_versions()

DetailsINSTALLED VERSIONS ------------------ commit: None python: 3.7.3.final.0 python-bits: 64 OS: Linux OS-release: 5.0.0-16-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8

pandas: 0.24.2
pytest: 4.5.0
pip: 19.1.1
setuptools: 41.0.1
Cython: 0.29.7
numpy: 1.16.3
scipy: 1.2.1
pyarrow: None
xarray: 0.12.1
IPython: 7.2.0
sphinx: None
patsy: None
dateutil: 2.8.0
pytz: 2019.1
blosc: None
bottleneck: None
tables: 3.5.1
numexpr: 2.6.9
feather: None
matplotlib: 3.0.3
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10.1
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None