ENH: Support large string types in the interchange protocol · Issue #56702 · pandas-dev/pandas (original) (raw)

Skip to content

Navigation Menu

• • GitHub Copilot Write better code with AI
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
• Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
• • Enterprise platform AI-powered developer platform
• Pricing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Appearance settings

Description

Feature Type

• Adding new functionality to pandas
• Changing existing functionality in pandas
• Removing existing functionality in pandas

Problem Description

The interchange protocol supports the large string type. Pandas could also support it, but currently does not. It only supports 'small' strings.

Feature Description

This currently works:

import pandas as pd

df = pd.Series([], name="a", dtype="string[pyarrow]").to_frame() dfi = df.dataframe() result = pd.api.interchange.from_dataframe(dfi)

result is equal to df

This does not:

import pandas as pd

df = pd.Series([], name="a", dtype="large_string[pyarrow]").to_frame() dfi = df.dataframe() result = pd.api.interchange.from_dataframe(dfi)

ValueError: Data type large_string[pyarrow] not supported by interchange protocol

Alternative Solutions

n/a

Additional Context

@MarcoGorelli FYI. Supporting large string would mean better interop with Polars and pyarrow.