ipc library

Apache Arrow IPC (columnar format V5) streams and files, shaped like PyArrow's pyarrow.ipc module. Import with a prefix:

import 'package:dartarrow/ipc.dart' as ipc;

final writer = ipc.newStream(table.schema)..writeTable(table);
final bytes = writer.close();
final roundTrip = ipc.openStream(bytes).readAll();

Readers and writers work on in-memory bytes. Readers return zero-copy views over uncompressed bodies, decode LZ4_FRAME and ZSTD compressed bodies (LZ4 only on the web), support dictionary replacement and delta batches in streams, and are bounded by IpcLimits. Writers emit uncompressed little-endian V5 output.

Errors: malformed input raises FormatException; valid input using an unsupported feature (big-endian data, unknown types or codecs) raises UnsupportedError. Writers raise UnsupportedError for types without a standard IPC encoding (binding int128/uint128) and ArgumentError for invalid input such as a mismatched batch schema.

Classes

IpcLimits
Resource limits applied while decoding untrusted IPC input.
RecordBatchFileReader
Random-access reader of an Arrow IPC file.
RecordBatchFileWriter
In-memory Arrow IPC file writer created by newFile.
RecordBatchStreamReader
Sequential reader of an Arrow IPC stream.
RecordBatchStreamWriter
In-memory Arrow IPC stream writer created by newStream.

Functions

newFile(Schema schema) → RecordBatchFileWriter
Creates an Arrow IPC file writer for schema, like PyArrow's ipc.new_file. See newStream for rejected schemas.
newStream(Schema schema) → RecordBatchStreamWriter
Creates an Arrow IPC stream writer for schema, like PyArrow's ipc.new_stream. Bytes are collected in memory and returned by RecordBatchStreamWriter.close.
openFile(Uint8List bytes, {IpcLimits limits = const IpcLimits()}) → RecordBatchFileReader
Opens an in-memory Arrow IPC file (ARROW1 magic plus footer), like PyArrow's ipc.open_file.
openStream(Uint8List bytes, {IpcLimits limits = const IpcLimits()}) → RecordBatchStreamReader
Opens an in-memory Arrow IPC stream, like PyArrow's ipc.open_stream.