BUG: SQL writing can lead to data loss · Issue #6509 · pandas-dev/pandas (original) (raw)

to_sql uses iterrows which leads to data conversion, and in the case of mixed data types, can lead to data loss.

s1 = pd.Series([0],dtype=np.float32) s2 = pd.Series([2**27 + 1],dtype=np.int32) df = pd.DataFrame({'float':s1, 'int':s2})

for row in df.iterrows(): print row[1][1] - (2**27 + 1)

(The same issue applies when using df.to_sql)

I found the same bug in to_stata and have submitted a PR on it.

to_sql is the only other location in pandas the uses iterrows, which should probably be avoided in all non-user code.

It should be an easy fix using something like itertuple - I don't use the SQL code so I'm not in a good position to fix this problem.