BUG: Float Nullable dtypes not being returned when use_nullable_dtypes=True in pd.read_parquet · Issue #45694 · pandas-dev/pandas (original) (raw)

Pandas version checks

Reproducible Example

import pandas as pd df = pd.DataFrame({'a':[1, 2, 3], 'b':[1.0, 2.0, 3]}) df['c'] = pd.Series([1, 2, 3], dtype='uint8') df.to_parquet('a') pd.read_parquet('a').dtypes a int64 b float64 c uint8 dtype: object pd.read_parquet('a', use_nullable_dtypes=True).dtypes a Int64 b float64 c UInt8 dtype: object pd.version '1.4.0'

Issue Description

When use_nullable_dtypes=True in parquet reader the floating column is being read as float64 instead of Float64(A nullable dtype)

Expected Behavior

pd.read_parquet('a', use_nullable_dtypes=True).dtypes
a Int64
b Float64
c UInt8
dtype: object

Installed Versions

pd.show_versions()
Traceback (most recent call last):
File "", line 1, in
File "/Users/pgali/opt/anaconda3/envs/del/lib/python3.9/site-packages/pandas/util/_print_versions.py", line 109, in show_versions
deps = _get_dependency_info()
File "/Users/pgali/opt/anaconda3/envs/del/lib/python3.9/site-packages/pandas/util/_print_versions.py", line 88, in _get_dependency_info
mod = import_optional_dependency(modname, errors="ignore")
File "/Users/pgali/opt/anaconda3/envs/del/lib/python3.9/site-packages/pandas/compat/_optional.py", line 126, in import_optional_dependency
module = importlib.import_module(name)
File "/Users/pgali/opt/anaconda3/envs/del/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/Users/pgali/opt/anaconda3/envs/del/lib/python3.9/site-packages/setuptools/init.py", line 8, in
import _distutils_hack.override # noqa: F401
File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
File "/Users/pgali/opt/anaconda3/envs/del/lib/python3.9/site-packages/_distutils_hack/override.py", line 1, in
import('_distutils_hack').do_override()
File "/Users/pgali/opt/anaconda3/envs/del/lib/python3.9/site-packages/_distutils_hack/init.py", line 71, in do_override
ensure_local_distutils()
File "/Users/pgali/opt/anaconda3/envs/del/lib/python3.9/site-packages/_distutils_hack/init.py", line 59, in ensure_local_distutils
assert '_distutils' in core.file, core.file
AssertionError: /Users/pgali/opt/anaconda3/envs/del/lib/python3.9/distutils/core.py