Icebird: JavaScript Iceberg Client

Icebird is a JavaScript client for Apache Iceberg tables. It reads and writes Iceberg v1/v2/v3 tables, runs SQL queries over them, and speaks to file-based or REST catalogs. It is built on top of hyparquet and hyparquet-writer for the underlying parquet I/O.

Usage

To read an Iceberg table:

const { icebergRead } = await import('icebird')

const tableUrl = 'https://s3.amazonaws.com/hyperparam-iceberg/spark/bunnies'
const data = await icebergRead({
  tableUrl,
  rowStart: 0,
  rowEnd: 10,
})

To read the Iceberg metadata (schema, etc):

import { icebergMetadata } from 'icebird'

const metadata = await icebergMetadata({ tableUrl })

// subsequent reads will be faster if you provide the metadata:
const data = await icebergRead({
  tableUrl,
  metadata,
})

Demo

Check out a minimal iceberg table viewer demo that shows how to integrate Icebird into a react web application using HighTable to render the table data. You can view any publicly accessible Iceberg table:

• Live Demo: https://hyparam.github.io/demos/icebird/
• Demo Source Code: https://github.com/hyparam/demos/tree/master/icebird

Time Travel

To fetch a previous version of the table, you can specify metadataFileName:

import { icebergRead } from 'icebird'

const data = await icebergRead({
  tableUrl,
  metadataFileName: 'v1.metadata.json',
})

Authentication

To add authentication or other custom fetch options, create a resolver and lister with requestInit and pass those into the public APIs:

import { icebergMetadata, icebergRead, s3Lister, urlResolver } from 'icebird'

const requestInit = {
  headers: {
    Authorization: 'Bearer my_token',
  },
}

const resolver = urlResolver({ requestInit })
const lister = s3Lister({ requestInit })

const metadata = await icebergMetadata({
  tableUrl,
  resolver,
  lister,
})

const data = await icebergRead({
  tableUrl,
  metadata,
  resolver,
  lister,
})

For private S3-compatible buckets (AWS, Cloudflare R2, MinIO), use s3SignedResolver which signs SigV4 via Web Crypto so it works in browsers and Node:

import { icebergRead, s3SignedResolver } from 'icebird'

const resolver = s3SignedResolver({
  accessKeyId, secretAccessKey, region: 'us-east-1',
  // For R2/MinIO, set endpoint and pathStyle:
  // endpoint: 'https://<acct>.r2.cloudflarestorage.com', pathStyle: true,
})
const data = await icebergRead({ tableUrl: 's3://my-bucket/warehouse/orders', resolver })

REST Catalog

For tables behind an Iceberg REST Catalog, connect via restCatalogConnect and pass the loaded metadata into icebergRead. Multi-level namespaces are arrays.

import { icebergRead, restCatalogConnect, restCatalogLoadTable } from 'icebird'

const ctx = await restCatalogConnect({ url: 'https://catalog.example.com' })
const { metadata } = await restCatalogLoadTable(ctx, { namespace: 'analytics', table: 'orders' })
const data = await icebergRead({ tableUrl: metadata.location, metadata })

SQL

Icebird ships a SQL engine on top of squirreling. icebergQuery runs a SQL query across one or more iceberg tables. Rows are streamed lazily. Multi-segment namespaces in the SQL FROM clause must be dot-separated and quoted: FROM "analytics.orders" resolves to namespace analytics, table orders.

import { collect, icebergQuery, restCatalogConnect } from 'icebird'

const catalog = await restCatalogConnect({ url: 'https://catalog.example.com' })
const result = await icebergQuery({
  catalog,
  query: 'SELECT "Breed Name", "Popularity Rank" FROM "java.bunnies" WHERE "Popularity Rank" <= 3 ORDER BY "Popularity Rank"',
})
const rows = await collect(result)

Writing

Icebird has experimental write support for Iceberg v2 (and v3 deletion vectors). All write functions take a Catalog and dispatch internally — the same call works against fileCatalog({ resolver }) or a REST catalog context returned by restCatalogConnect.

import {
  fileCatalog,
  icebergAppend,
  icebergCreateTable,
  icebergDelete,
  icebergExpireSnapshots,
  icebergRewrite,
  icebergSetRef,
} from 'icebird'

// `urlResolver()` ships with a `writer` (HTTP PUT) and `deleter` (HTTP DELETE);
// pass a custom `requestInit` to it for auth headers. For non-HTTP backends,
// supply your own `Resolver` with `writer` and (for drop) `deleter`.
const catalog = fileCatalog({ resolver })
const tableUrl = 's3://my-bucket/warehouse/orders'

const schema = {
  type: 'struct',
  'schema-id': 0,
  fields: [
    { id: 1, name: 'id', required: true, type: 'long' },
    { id: 2, name: 'name', required: false, type: 'string' },
  ],
}

await icebergCreateTable({ catalog, tableUrl, schema })
await icebergAppend({ catalog, tableUrl, records: [{ id: 1n, name: 'alice' }] })

// position deletes — v3 writes deletion vectors; v2 writes parquet delete files
await icebergDelete({
  catalog, tableUrl,
  deletes: [{ file_path: 's3://.../data/abc.parquet', pos: 0 }],
})

// snapshot management
await icebergSetRef({ catalog, tableUrl, ref: 'main', snapshotId })
await icebergExpireSnapshots({ catalog, tableUrl, snapshotIds: [oldSnapshotId] })

If the table is created with a sortOrder, icebergAppend orders the rows in each written file by that order (tightening per-file column bounds for scan pruning). icebergRewrite compacts the current snapshot — reading every live row (deletes applied), sorting globally, and rewriting into consolidated, non-overlapping files via a replace snapshot (v2 tables):

// compact small files into sorted, non-overlapping ones
await icebergRewrite({ catalog, tableUrl })
// optionally split large partitions and/or re-partition under another spec
await icebergRewrite({ catalog, tableUrl, targetFileRows: 1_000_000, partitionSpecId: 1 })

A rewrite is not retried on a concurrent commit (it would risk dropping rows another writer appended meanwhile); on conflict it throws and should be re-run against fresh metadata.

For a REST catalog, swap fileCatalog(...) for the connect context and pass namespace/table instead of tableUrl:

const catalog = await restCatalogConnect({ url: 'https://catalog.example.com' })
await icebergAppend({ catalog, namespace: 'analytics', table: 'orders', records })

icebergDropTable on a file catalog requires a lister to enumerate files; pass purgeRequested: true to also delete data/.

Supported Features

Icebird aims to support reading any Iceberg table, but currently only supports a subset of the features. The following features are supported:

Feature	Supported	Notes
Read Iceberg v1 Tables	✅
Read Iceberg v2 Tables	✅
Read Iceberg v3 Tables	✅
Write Iceberg v2 Tables	✅
Write Iceberg v3 Tables	✅
Parquet Storage	✅
Avro Storage	✅
ORC Storage	❌
Puffin Storage	⚠️	Supports uncompressed deletion-vector-v1 blobs only.
File-based Catalog (version-hint.text)	✅
REST Catalog	✅
Hive Catalog	❌
Glue Catalog	❌
Service-based Catalog	❌
Position Deletes	✅	Supports Parquet position delete files and Puffin deletion vectors.
Equality Deletes	✅
Binary Deletion Vectors	✅	Supports uncompressed Puffin deletion-vector-v1 blobs.
Delete Partition Scope	✅	Applies sequence and partition scope before filtering rows.
Rename Columns	✅
All Parquet Compression Codecs	✅
All Parquet Types	✅
Variant Types	✅
Geometry Types	✅
Geography Types	✅
Row Lineage	✅	v3 _row_id and _last_updated_sequence_number inheritance.
Sorting	✅	Orders rows by the declared sort order on append; icebergRewrite compacts to sorted, non-overlapping files (v2).
Scan Pruning	✅	Skips data files via partition tuples and manifest column bounds, and parquet row groups via column statistics.
Encryption	❌

References

• https://iceberg.apache.org/spec/
• https://avro.apache.org/docs/1.12.0/specification/
• https://github.com/hyparam/hyparquet
• https://github.com/hyparam/hyparquet-writer
• https://github.com/apache/iceberg
• https://github.com/apache/iceberg-python
• https://github.com/apache/iceberg-rust