BUG: SparseDataFrame from scipy.sparse.dok_matrix returns incorrect dataframe in python 2.7 · Issue #16179 · pandas-dev/pandas (original) (raw)

Code Sample, a copy-pastable example if possible

Python 2.7

import sys sys.version '2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:05:08) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]' import numpy as np import scipy.sparse a = np.arange(1, 5).reshape(2,2) a array([[1, 2], [3, 4]]) spm = scipy.sparse.dok_matrix(a) spm <2x2 sparse matrix of type '<type 'numpy.int64'>' with 4 stored elements in Dictionary Of Keys format> pd.SparseDataFrame(a) 0 1 0 1 2 1 3 4 pd.SparseDataFrame(spm) 0 1 0 3 2 1 1 4

Python 3.6

import sys sys.version '3.6.1 |Continuum Analytics, Inc.| (default, Mar 22 2017, 19:25:17) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]' a = np.arange(1, 5).reshape(2,2) spm = scipy.sparse.dok_matrix(a) pd.SparseDataFrame(spm) 0 1 0 1 2 1 3 4

Problem description

Initialization of SparseDataFrame with scipy.sparse.dok_matrix returns a different result from direct initialization from np.ndarray only in Python 2.7. This is inconsistent and unestimated behavior.
The scipy.sparse.dia_matrix shows similar buggy behavior.

Expected Output

spm = scipy.sparse.dok_matrix(np.arange(1, 5).reshape(2,2)) pd.SparseDataFrame(spm) 0 1 0 1 2 1 3 4

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: 075eca1python: 2.7.13.final.0 python-bits: 64 OS: Darwin OS-release: 16.5.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: ja_JP.UTF-8 LOCALE: None.None

pandas: 0.20.0rc1+29.g075eca1
pytest: 3.0.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.18.1
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
pandas_gbq: None
pandas_datareader: None