BUG: Sparse concat may fill fill_value with NaN by sinhrks · Pull Request #12966 · pandas-dev/pandas (original) (raw)

# on current master
dense1 = pd.DataFrame({'A': [1, 2, 3, np.nan],
                    'B': [0, 0, 0, 0],
                    'C': [np.nan, np.nan, np.nan, np.nan],
                    'D': [1, 2, 3, 4]})
dense2 = pd.DataFrame({'E': [1, 2, 3, np.nan],
                    'F': [0, 0, 0, 0],
                    'G': [np.nan, np.nan, np.nan, np.nan],
                    'H': [1, 2, 3, 4]})
sparse1 = dense1.to_sparse()
sparse2 = dense2.to_sparse()
pd.concat([sparse2, sparse1], axis=1)
# AttributeError: 'int' object has no attribute 'ravel'

One point to be discussed is the logic for return type. Currently, SparseDataFrame is returned only when all blocks are all sparse. Because SparseDataFrame can't work properly if dense block is contained.

Thus, dense and sparse concat with axis=0 resunts in SparseDataFrame, and axis=1 results in normal DataFrame.