ENH: sql support · Issue #4163 · pandas-dev/pandas (original) (raw)

Skip to content

Navigation Menu

• • GitHub Copilot Write better code with AI
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
• Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
• • Enterprise platform AI-powered developer platform
• Pricing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Appearance settings

Description

UPDATE: this is continued in #6292 after the refactor based on sqlalchemy.

This is a central issue to track sql support (migrated from the mysql support ticket - #2482 and #1662).

• sqlite
• mysql (mysql support #2482)
• oracle (not tested, partially working)
• postgres, (partial via ENH: add PostgreSQL support for pandas.io.sql.get_schema #4574) (not tested, some parts possibly working, [SO user suggests it "works"],(http://stackoverflow.com/questions/17156084/unpacking-a-sql-select-into-a-pandas-dataframe/17156233#17156233))
• ODBC
• any others?

I think we are agreed we will be moving to use SQLAlchemy, in the hope of comprehensive cross-platform support:

• SQLAlchemy (SQLAlchemy in io.sql to manage different SQL dialects #2717, SQLalchemy interface in pandas.io #191)

Todo:

• travis should not skip tests (e.g. mysql TST: Do not skip MySQL tests on Travis. #4177)
• add flavor argument to read_sql
• handling no row returns (BUG: handle no rows returns from a sql query #3745)
• Specify chunksize (and iterator) for read_frame (Feature request: specify chunksize for read_sql #2908)
• read and write the index (and multi-index)
• mangle_dup_columns for read_frame (Handling of duplicate columns in pandas.io.sql.read_frame #2738)
• bug in write_frame replace (BUG: in io.sql.write_frame (replace) #4110, BUG: io.sql.write_frame(if_exists='replace') not working as expected (with fix) #2971, PR BUG/TST: Fix io.sql.write_frame replace bug and complete test coverage of if_exists functionality #4304, 4304r if_exists == 'fail' should actually work #6164)
• option in config to specify default engine/connection for SQL (maybe?)
• refactor SQLAlchemy/sqliite tests (@hayd was thinking different flavors could inherit tests from a baseclass, so tests are only flavor explicit if obviously flavor explicit...)

Better Dtype coercion:

• infer dtypes more carefully (discussed in mysql support #2482)
• NaN/NaT conversions (ENH: sql support for NaN/NaT conversions #2754, NaN value in dataframe caused _write_mysql error #4199, PR BUG: convert nan to None before insert data into mysql #4200)
• datetime.date conversions https://groups.google.com/forum/#!topic/pydata/KEBKxj4cWNI
• support encoding Add option for default_charset to sql.write_frame #5008

Peformance

• Vbenches! (for various flavors even?)

cc @garaud @mangecoeur @danielballan @changhiskhan

Metadata

Metadata

Labels

Development

No branches or pull requests

Issue actions