ENH: add gzip/bz2 compression to relevant read_* methods · Issue #15644 · pandas-dev/pandas (original) (raw)

Skip to content

Navigation Menu

• • GitHub Copilot Write better code with AI
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
• Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
• • Enterprise platform AI-powered developer platform
• Pricing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Description

This issue is a branch off of #11666, which implemented compression support for read_pickle. There are still a few other read_* methods that could possibly benefit from compression support. Looking at the I/O API reference, this jump out at me:

• read_json - This can definitely benefit from compression. I've stored very large gzipped JSON files before. As a general rule, any read_* method that supports any kind of plaintext format should support compression.
• read_stata- I don't use Stata, but it looks like a .dta file is not a plaintext file. Is it naturally compressed, or can they be compressed significantly like pickles?
• read_sas - I've also never used SAS, and like Stata's .dta files, it looks like .xpt and .sas7bdat files are both some binary format. Can they be compressed well?