Semi-Structured Data in Apache Iceberg: Meet the Variant Type
Semi-structured data, such as JSON-like documents whose fields differ from row to row, has always been a poor fit for table formats built around fixed schemas. Iceberg v3 adds the Variant type for exactly this data: a single column can hold values of arbitrary, evolving shape, stored in a compact binary form that engines read and write consistently.
This is the first post in a series on Variant in Apache Iceberg. It covers what Variant is, why it exists, and how it fits into an Iceberg table. The next post covers shredding, the technique that stores frequently accessed Variant fields as typed, columnar data. Variant is stored in Parquet, Avro, and ORC; shredding is currently available only in Parquet.