pd.to/read_sql_table silently corrupts Categorical columns · Issue #8624 · pandas-dev/pandas (original) (raw)

Skip to content

Navigation Menu

• • GitHub Copilot Write better code with AI
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
• Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
• • Enterprise platform AI-powered developer platform
• Pricing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Appearance settings

Description

In [29]: from sqlalchemy import create_engine ...: engine = create_engine('sqlite://') ...: df=pd.DataFrame([[1,'John P. Doe'],[2,'Jane Dove'],[1,'John P. Doe']], ...: columns=['person_id','person_name']) ...: df.to_sql('data1',engine) ...: df['person_name']=pd.Categorical(df.person_name) ...: df.to_sql('data2',engine) ...: print pd.read_sql_table('data1',engine) ...: print pd.read_sql_table('data2',engine) index person_id person_name 0 0 1 John P. Doe 1 1 2 Jane Dove 2 2 1 John P. Doe index person_id person_name 0 0 1 J 1 1 2 o 2 2 1 h

using relational db to store catagorical columns in seperare tables would be very cool, and rebuilding the frame in pandas by JOIN from the multiple tables would save time on the wire. also memory if categorical was build directly.