ROADMAP: Consistent missing value handling with new NA scalar · Issue #28095 · pandas-dev/pandas (original) (raw)

Skip to content

Navigation Menu

• • GitHub Copilot Write better code with AI
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
• Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
• • Enterprise platform AI-powered developer platform
• Pricing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Description

I cleaned up my initial write up on the consistent missing values proposal (#27825 (comment)), and incorporated the items brought up in the last video chat. So I think it is ready for some more detailed discussion.

The last version of the full proposal can be found here: https://hackmd.io/@jorisvandenbossche/Sk0wMeAmB

TL;DR:

• I propose to introduce a new scalar (singleton) pd.NA that can be used as the missing value indicator (when accessing a single value, not necessarily how it is stored under the hood).
• This can be used instead of np.nan or pd.NaT in new data types (eg nullable integers, potential string dtype)
• Long term, we can see if there is a migration possible to use this consistently for all data types.

cc @pandas-dev/pandas-core