"Bad" lines with too few fields · Issue #9729 · pandas-dev/pandas (original) (raw)

Skip to content

Navigation Menu

• • GitHub Copilot Write better code with AI
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
• Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
• • Enterprise platform AI-powered developer platform
• Pricing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Appearance settings

Description

As per http://pandas.pydata.org/pandas-docs/stable/io.html#handling-bad-lines, records with too many fields cause (depending on error_bad_lines) an exception to be thrown, or stderr to be written to.

Would it be possible to add an option, defaulting to False, that gave similar warnings/errors if there are too few fields in a record (compared to all the other records. The current behaviour is just to insert NaN's - but there are cases where data integrity is important so knowing that some records are missing fields is important.

Cheers ^