PERF: allow to skip validation/sanitization in DataFrame._from_arrays by jorisvandenbossche · Pull Request #32858 · pandas-dev/pandas (original) (raw)

In [1]: arrays = [pd.arrays.SparseArray(np.random.randint(0, 2, 1000), dtype="float64") for _ in range(10000)] 
   ...: index = pd.Index(range(len(arrays[0])))   
   ...: columns = pd.Index(range(len(arrays)))

In [2]: %timeit pd.DataFrame._from_arrays(arrays, index=index, columns=columns)   
119 ms ± 3.52 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [3]: %timeit pd.DataFrame._from_arrays(arrays, index=index, columns=columns, verify_integrity=False)    
98.1 ms ± 713 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)