pyarrow.input_stream — Apache Arrow v20.0.0 (original) (raw)

pyarrow.input_stream(source, compression='detect', buffer_size=None)#

Create an Arrow input stream.

Parameters:

sourcestr, Path, buffer, or file-like object

The source to open for reading.

compressionstr optional, default ‘detect’

The compression algorithm to use for on-the-fly decompression. If “detect” and source is a file path, then compression will be chosen based on the file extension. If None, no compression will be applied. Otherwise, a well-known algorithm name must be supplied (e.g. “gzip”).

buffer_sizeint, default None

If None or 0, no buffering will happen. Otherwise the size of the temporary read buffer.

Examples

Create a readable BufferReader (NativeFile) from a Buffer or a memoryview object:

import pyarrow as pa buf = memoryview(b"some data") with pa.input_stream(buf) as stream: ... stream.read(4) ... b'some'

Create a readable OSFile (NativeFile) from a string or file path:

import gzip with gzip.open('example.gz', 'wb') as f: ... f.write(b'some data') ... 9 with pa.input_stream('example.gz') as stream: ... stream.read() ... b'some data'

Create a readable PythonFile (NativeFile) from a a Python file object:

with open('example.txt', mode='w') as f: ... f.write('some text') ... 9 with pa.input_stream('example.txt') as stream: ... stream.read(6) ... b'some t'