dartarrow

Install

Not yet on pub.dev. Use a Git dependency, and pin ref to a full commit SHA for reproducible builds.

dependencies:
  dartarrow:
    git:
      url: https://github.com/richardrh/dartarrow.git
      ref: main

Usage

import 'package:dartarrow/arrow.dart' as pa;

final values = pa.array([10, null, 30], type: pa.int32());
values[1].isValid;              // false: a typed int32 null Scalar
values[0].asDart();             // 10
values.slice(1, 2).toDartList(); // [null, 30]

final schema = pa.schema([pa.field('value', pa.int32())]);
final batch = pa.RecordBatch.fromArrays([values], schema: schema);
final table = pa.table({
  'value': pa.chunkedArray([values.slice(0, 1), values.slice(1)]),
});
table.column('value').numChunks; // 2

Batch columns are Array; table columns are always ChunkedArray, and chunk boundaries survive until combineChunks().
slice(offset, length) takes a length, not an end index.
Field and Schema keep duplicate and empty names. Metadata is bytes-keyed; default equality ignores it.
isValid() and isNull() on an array return Boolean masks. For one element use array[i].isValid.

Precision

Conversion never loses precision. Wide integers become BigInt. Decimals keep an unscaled BigInt and scale. Scalar.value on temporal types is the raw counter; asDart() returns a UTC DateTime or Duration when exactly representable and a TemporalValue otherwise. NaN is valid, not null.

Iteration

Array does not implement Iterable, because its generic cast<R>() clashes with Arrow's cast(type). Use for (final s in values.scalars) or values.toDartList().

JSON

json.dart provides OwnedBatchJsonCodec, a package-specific JSON format. It is not Arrow IPC or Arrow integration JSON.

Types

Family	Types
Primitive	null, bool, int8–64, uint8–64, float16/32/64
Decimal	decimal32/64/128/256, including negative scales
Text and binary	utf8, binary, large offsets, fixed-size binary, string/binary views
Temporal	date, time, timestamp (with timezone), duration, year-month / day-time / month-day-nano intervals
Nested	list, large list, list view, fixed-size list, struct, map
Encoded	dictionary (including nested), run-end encoded, sparse and dense union, extension

Not included: compute kernels, datasets, Parquet and CSV. bindingInt128() and bindingUint128() are binding extensions, not Arrow types, and have no IPC encoding.

Memory

Building arrays from Dart values copies them into owned buffers. Array.fromBuffers borrows instead. Slices are constant-time views over the same buffers.

final bytes = Uint8List(8);
ByteData.sublistView(bytes)
  ..setInt32(0, 10, Endian.little)
  ..setInt32(4, 20, Endian.little);
final borrowed = pa.Array.fromBuffers(pa.int32(), 2, [null, pa.Buffer(bytes)]);

Keep every backing allocation alive while the array or any slice, child or dictionary is in use.
Do not mutate the backing bytes while they are borrowed. Read-only views cannot prevent this.
Buffer 0 is the optional validity bitmap (LSB-first). null means all valid.

C ABI

package:dartarrow/arrow_c.dart declares the standard CArrowSchema, CArrowArray and CArrowArrayStream structs for dart:ffi. They are layouts only. Callers manage allocation and release according to the C Data Interface. The core arrow.dart does not import FFI.