Semi-Structured Data in Apache Iceberg: Meet the Variant Type
Semi-structured data, such as JSON-like documents whose fields differ from row to row, has always been a poor fit for table formats built around fixed schemas. Iceberg v3 adds the Variant type for exactly this data: a single column can hold values of arbitrary, evolving shape, stored as a compact binary value that engines read and write consistently.
This is the first post in a series on Variant in Apache Iceberg: what Variant is, why it exists, and how it fits into an Iceberg table. Parquet already defines the Variant type and its binary encoding, so this post focuses on what Iceberg adds on top: how Variant fits the table's schema, files, snapshots, and statistics.
Variant is stored in Parquet, Avro, and ORC. Shredding, an optimization that stores commonly queried Variant fields as their own typed columns, is available only in Parquet; a later post covers how it works.