pyarrow.dataset.Partitioning — Apache Arrow v20.0.0 (original) (raw)

class pyarrow.dataset.Partitioning#

Bases: _Weakrefable

__init__(*args, **kwargs)#

Methods

Attributes

format(self, expr)#

Convert a filter expression into a tuple of (directory, filename) using the current partitioning scheme

Parameters:

exprpyarrow.dataset.Expression

Returns:

tuple[str, str]

Examples

Specify the Schema for paths like “/2009/June”:

import pyarrow as pa import pyarrow.dataset as ds import pyarrow.compute as pc part = ds.partitioning(pa.schema([("year", pa.int16()), ... ("month", pa.string())])) part.format( ... (pc.field("year") == 1862) & (pc.field("month") == "Jan") ... ) ('1862/Jan', '')

parse(self, path)#

Parse a path into a partition expression.

Parameters:

pathstr

Returns:

pyarrow.dataset.Expression

schema#

The arrow Schema attached to the partitioning.