read_csv(engine='c') can insert spurious rows full of NaNs · Issue #10022 · pandas-dev/pandas (original) (raw)

Skip to content

Navigation Menu

• • GitHub Copilot Write better code with AI
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
• Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
• • Enterprise platform AI-powered developer platform
• Pricing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Description

I have a well-formed CSV file with about 70k lines that parses out to a DataFrame with about 170k rows, where the extra rows are just full of NaNs. It only happens with the 'c' engine.

This is with git master.

I won't bother to upload the CSV file in question because I think I've tracked down the problem. It seems that tokenize_delimited() runs on chunks of data at a time. This problem occurs when a chunk happens to start with '\n'. I'll send a pull request.