OverflowError for string of large numbers (int64) while reading json · Issue #18842 · pandas-dev/pandas (original) (raw)

import json import pandas as pd json_data = {"articleId":"1404366058080022500245"} pd.read_json(json.dumps(json_data), typ='series')

Python int too large to convert to C long: OverflowError ... File "/var/task/pandas/io/json/json.py", line 366, in read_json return json_reader.read() File "/var/task/pandas/io/json/json.py", line 467, in read obj = self._get_object_parser(self.data) File "/var/task/pandas/io/json/json.py", line 489, in _get_object_parser obj = SeriesParser(json, **kwargs).parse() File "/var/task/pandas/io/json/json.py", line 582, in parse self._try_convert_types() File "/var/task/pandas/io/json/json.py", line 752, in _try_convert_types 'data', self.obj, convert_dates=self.convert_dates) File "/var/task/pandas/io/json/json.py", line 621, in _try_convert_data new_data, result = self._try_convert_to_date(data) File "/var/task/pandas/io/json/json.py", line 684, in _try_convert_to_date new_data = data.astype('int64') File "/var/task/pandas/util/_decorators.py", line 118, in wrapper return func(*args, **kwargs) File "/var/task/pandas/core/generic.py", line 4004, in astype **kwargs) File "/var/task/pandas/core/internals.py", line 3462, in astype return self.apply('astype', dtype=dtype, **kwargs) File "/var/task/pandas/core/internals.py", line 3329, in apply applied = getattr(b, f)(**kwargs) File "/var/task/pandas/core/internals.py", line 544, in astype **kwargs) File "/var/task/pandas/core/internals.py", line 625, in _astype values = astype_nansafe(values.ravel(), dtype, copy=True) File "/var/task/pandas/core/dtypes/cast.py", line 692, in astype_nansafe return lib.astype_intsafe(arr.ravel(), dtype).reshape(arr.shape) File "pandas/_libs/lib.pyx", line 854, in pandas._libs.lib.astype_intsafe File "pandas/_libs/src/util.pxd", line 91, in util.set_value_at_unsafe OverflowError: Python int too large to convert to C long

Reading json containing stringified numbers larger than int64 causes OverflowError in pandas 0.21.1. Appears to be working fine in 0.21.0

Max