Implementation Status🔗
Apache Iceberg's table specification is implemented in multiple languages. This page provides an overview of the current capabilities.
Libraries🔗
This section lists the libraries that implement the Apache Iceberg specification.
| Library | Released Version |
|---|---|
| Java | 1.12.0 |
| PyIceberg | 0.12.0 |
| Rust | 0.11.0 |
| Go | 0.7.0 |
| C++ | 0.4.0 |
Unless stated otherwise, Y means the released version listed above supports the capability and N means it does not.
For file formats, Y requires both reading and writing. Support available only on main is N. A tab labeled with a version
range, such as "V1 - V3", means the same capabilities apply to each version in the range.
Data Types🔗
Y means the library supports the type in schemas and can both read and write its values in at least one supported data file
format. N means one or more of these capabilities is missing. For example, recognizing geometry in a schema without being
able to read and write geometry values is N. These cells describe data file IO. Committing those files to a table requires
the support listed under Table Write Operations.
Table Spec🔗
| Data Type | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| boolean | Y | Y | Y | Y | Y |
| int | Y | Y | Y | Y | Y |
| long | Y | Y | Y | Y | Y |
| float | Y | Y | Y | Y | Y |
| double | Y | Y | Y | Y | Y |
| decimal | Y | Y | Y | Y | Y |
| date | Y | Y | Y | Y | Y |
| time | Y | Y | Y | Y | Y |
| timestamp | Y | Y | Y | Y | Y |
| timestamptz | Y | Y | Y | Y | Y |
| string | Y | Y | Y | Y | Y |
| uuid | Y | Y | Y | Y | Y |
| fixed | Y | Y | Y | Y | Y |
| binary | Y | Y | Y | Y | Y |
| list | Y | Y | Y | Y | Y |
| map | Y | Y | Y | Y | Y |
| struct | Y | Y | Y | Y | Y |
| Data Type | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| boolean | Y | Y | Y | Y | Y |
| int | Y | Y | Y | Y | Y |
| long | Y | Y | Y | Y | Y |
| float | Y | Y | Y | Y | Y |
| double | Y | Y | Y | Y | Y |
| decimal | Y | Y | Y | Y | Y |
| date | Y | Y | Y | Y | Y |
| time | Y | Y | Y | Y | Y |
| timestamp | Y | Y | Y | Y | Y |
| timestamptz | Y | Y | Y | Y | Y |
| timestamp_ns | Y | Y | Y | Y | Y |
| timestamptz_ns | Y | Y | Y | Y | Y |
| unknown | Y | Y | N | Y | Y |
| string | Y | Y | Y | Y | Y |
| uuid | Y | Y | Y | Y | Y |
| fixed | Y | Y | Y | Y | Y |
| binary | Y | Y | Y | Y | Y |
| variant | Y | N | N | Y | N |
| geometry | Y | N | N | Y | Y |
| geography | Y | N | N | Y | Y |
| list | Y | Y | Y | Y | Y |
| map | Y | Y | Y | Y | Y |
| struct | Y | Y | Y | Y | Y |
Table Metadata and Features🔗
Table Spec🔗
| Capability | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Read table metadata | Y | Y | Y | Y | Y |
| Write table metadata | Y | Y | Y | Y | Y |
| Capability | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Read table metadata | Y | Y | Y | Y | Y |
| Write table metadata | Y | N | Y | Y | Y |
| Read initial column defaults | Y | Y | Y | Y | Y |
| Read multi-argument transforms | N | N | N | N | N |
| Write multi-argument transforms | N | N | N | N | N |
| Read row lineage columns | Y | N | Y | Y | Y |
| Write row lineage | Y | N | Y | Y | Y |
| Read encrypted tables | Y | N | Y | N | N |
| Write encrypted tables | Y | N | Y | N | N |
Data File Formats🔗
| Format | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Parquet | Y | Y | Y | Y | Y |
| ORC | Y | N | N | N | N |
| Puffin | Y | N | Y | Y | Y |
| Avro | Y | N | N | N | Y |
File IO🔗
| Storage | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Local Filesystem | Y | Y | Y | Y | Y |
| Hadoop Filesystem | Y | Y | N | N | N |
| S3 Compatible | Y | Y | Y | Y | Y |
| GCS Compatible | Y | Y | Y | Y | N |
| ADLS Compatible | Y | Y | Y | Y | N |
Table Maintenance Operations🔗
Table Spec🔗
| Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Update schema | Y | Y | Y | Y | Y |
| Update partition spec | Y | Y | N | Y | Y |
| Update table properties | Y | Y | Y | Y | Y |
| Replace sort order | Y | Y | Y | Y | Y |
| Update table location | Y | N | Y | Y | Y |
| Update statistics | Y | Y | Y | Y | Y |
| Update partition statistics | Y | N | N | Y | Y |
| Expire snapshots | Y | Y | Y | Y | Y |
| Manage snapshots | Y | Y | N | Y | Y |
| Operation | Java | PyIceberg1 | Rust | Go | C++ |
|---|---|---|---|---|---|
| Update schema | Y | Y | Y | Y | Y |
| Update partition spec | Y | Y | N | Y | Y |
| Update table properties | Y | Y | Y | Y | Y |
| Replace sort order | Y | Y | Y | Y | Y |
| Update table location | Y | N | Y | Y | Y |
| Update statistics | Y | Y | Y | Y | Y |
| Update partition statistics | Y | N | N | Y | Y |
| Expire snapshots | Y | Y | Y | Y | Y |
| Manage snapshots | Y | Y | N | Y | Y |
-
PyIceberg supports the operations marked
Ythrough a REST catalog whose server writes V3 metadata. SQL, Glue, and Hive catalogs cannot persist V3 metadata with PyIceberg 0.12.0. ↩
Table Update Operations🔗
Table Spec🔗
| Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Append data files | Y | Y | Y | Y | Y |
| Rewrite files | Y | N | N | Y | Y |
| Rewrite manifests | Y | N | N | Y | N |
| Overwrite files | Y | Y | N | Y | Y |
| Delete files | Y | Y | N | Y | Y |
| Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Append data files | Y | Y | Y | Y | Y |
| Rewrite files | Y | N | N | Y | Y |
| Rewrite manifests | Y | N | N | Y | N |
| Overwrite files | Y | Y | N | Y | Y |
| Delete files | Y | Y | N | Y | Y |
| Row delta | Y | N | N | Y | Y |
| Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Append data files | Y | N | Y | Y | Y |
| Rewrite files | Y | N | N | Y | Y |
| Rewrite manifests | Y | N | N | Y | N |
| Overwrite files | Y | N | N | Y | Y |
| Delete files | Y | N | N | Y | Y |
| Row delta | Y | N | N | Y | Y |
Table Read Operations🔗
Table Spec🔗
| Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Plan with data file | Y | Y | Y | Y | Y |
| Plan with puffin statistics | Y | N | N | N | N |
| Read data file | Y | Y | Y | Y | Y |
| Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Plan with data file | Y | Y | Y | Y | Y |
| Plan with position deletes | Y | Y | Y | Y | Y |
| Plan with equality deletes | Y | N | Y | Y | Y |
| Plan with puffin statistics | Y | N | N | N | N |
| Read data file | Y | Y | Y | Y | Y |
| Read with position deletes | Y | Y | Y | Y | Y |
| Read with equality deletes | Y | N | Y | Y | Y |
| Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Plan with data file | Y | Y | Y | Y | Y |
| Plan with position deletes | Y | Y | Y | Y | Y |
| Plan with equality deletes | Y | N | Y | Y | Y |
| Plan with deletion vectors | Y | Y | Y | Y | Y |
| Plan with puffin statistics | Y | N | N | N | N |
| Read data file | Y | Y | Y | Y | Y |
| Read with position deletes | Y | Y | Y | Y | Y |
| Read with equality deletes | Y | N | Y | Y | Y |
| Read with deletion vectors | Y | Y | Y | Y | Y |
Table Write Operations🔗
Table Spec🔗
| Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Append data | Y | Y | Y | Y | Y |
| Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Append data | Y | Y | Y | Y | Y |
| Write position deletes | Y | N | N | Y | Y |
| Write equality deletes | Y | N | N | Y | Y |
V3 writers must use deletion vectors instead of adding position delete files. Existing position deletes remain readable.
| Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Append data | Y | N | Y | Y | Y |
| Write equality deletes | Y | N | N | Y | Y |
| Write deletion vectors | Y | N | N | Y | Y |
Catalogs🔗
Rest Catalog🔗
Table Spec🔗
| Table Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listTable | Y | Y | Y | Y | Y |
| createTable | Y | Y | Y | Y | Y |
| dropTable | Y | Y | Y | Y | Y |
| loadTable | Y | Y | Y | Y | Y |
| updateTable | Y | Y | Y | Y | Y |
| renameTable | Y | Y | Y | Y | Y |
| tableExists | Y | Y | Y | Y | Y |
View Spec V1🔗
| View Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| createView | Y | Y | N | Y | N |
| dropView | Y | Y | N | Y | N |
| listView | Y | Y | N | Y | N |
| viewExists | Y | Y | N | Y | N |
| replaceView | Y | N | N | Y | N |
| renameView | Y | N | N | Y | N |
Namespace Operations🔗
| Namespace Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listNamespaces | Y | Y | Y | Y | Y |
| createNamespace | Y | Y | Y | Y | Y |
| dropNamespace | Y | Y | Y | Y | Y |
| namespaceExists | Y | Y | Y | Y | Y |
| updateNamespaceProperties | Y | Y | N | Y | Y |
| loadNamespaceMetadata | Y | Y | Y | Y | Y |
Sql Catalog🔗
The sql catalog is a catalog backed by a sql database, which is called jdbc catalog in java.
| Database | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| Postgres | Y | Y | Y | Y | Y |
| MySQL | Y | Y | Y | Y | Y |
| SQLite | Y | Y | Y | Y | Y |
Table Spec🔗
| Table Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listTable | Y | Y | Y | Y | Y |
| createTable | Y | Y | Y | Y | Y |
| dropTable | Y | Y | Y | Y | Y |
| loadTable | Y | Y | Y | Y | Y |
| updateTable | Y | Y | Y | Y | Y |
| renameTable | Y | Y | Y | Y | Y |
| tableExists | Y | Y | Y | Y | Y |
| Table Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listTable | Y | Y | Y | Y | Y |
| createTable | Y | N | Y | Y | Y |
| dropTable | Y | Y | Y | Y | Y |
| loadTable | Y | Y | Y | Y | Y |
| updateTable | Y | N | Y | Y | Y |
| renameTable | Y | Y | Y | Y | Y |
| tableExists | Y | Y | Y | Y | Y |
View Spec V1🔗
| View Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| createView | Y | N | N | Y | N |
| dropView | Y | N | N | Y | N |
| listView | Y | N | N | Y | N |
| viewExists | Y | N | N | Y | N |
| replaceView | Y | N | N | N | N |
| renameView | Y | N | N | N | N |
Namespace Operations🔗
| Namespace Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listNamespaces | Y | Y | Y | Y | Y |
| createNamespace | Y | Y | Y | Y | Y |
| dropNamespace | Y | Y | Y | Y | Y |
| namespaceExists | Y | Y | Y | Y | Y |
| updateNamespaceProperties | Y | Y | Y | Y | Y |
| loadNamespaceMetadata | Y | Y | Y | Y | Y |
Glue Catalog🔗
Table Spec🔗
| Table Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listTable | Y | Y | Y | Y | N |
| createTable | Y | Y | Y | Y | N |
| dropTable | Y | Y | Y | Y | N |
| loadTable | Y | Y | Y | Y | N |
| updateTable | Y | Y | Y | Y | N |
| renameTable | Y | Y | Y | Y | N |
| tableExists | Y | Y | Y | Y | N |
| Table Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listTable | Y | Y | Y | Y | N |
| createTable | Y | N | Y | Y | N |
| dropTable | Y | Y | Y | Y | N |
| loadTable | Y | Y | Y | Y | N |
| updateTable | Y | N | Y | Y | N |
| renameTable | Y | Y | Y | Y | N |
| tableExists | Y | Y | Y | Y | N |
View Spec V1🔗
| View Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| createView | N | N | N | N | N |
| dropView | N | N | N | N | N |
| listView | N | N | N | N | N |
| viewExists | N | N | N | N | N |
| replaceView | N | N | N | N | N |
| renameView | N | N | N | N | N |
Namespace Operations🔗
| Namespace Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listNamespaces | Y | Y | Y | Y | N |
| createNamespace | Y | Y | Y | Y | N |
| dropNamespace | Y | Y | Y | Y | N |
| namespaceExists | Y | Y | Y | Y | N |
| updateNamespaceProperties | Y | Y | Y | Y | N |
| loadNamespaceMetadata | Y | Y | Y | Y | N |
Hive Metastore Catalog🔗
Table Spec🔗
| Table Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listTable | Y | Y | Y | Y | N |
| createTable | Y | Y | Y | Y | N |
| dropTable | Y | Y | Y | Y | N |
| loadTable | Y | Y | Y | Y | N |
| updateTable | Y | Y | N | Y | N |
| renameTable | Y | Y | Y | Y | N |
| tableExists | Y | Y | Y | Y | N |
| Table Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listTable | Y | Y | Y | Y | N |
| createTable | Y | N | Y | Y | N |
| dropTable | Y | Y | Y | Y | N |
| loadTable | Y | Y | Y | Y | N |
| updateTable | Y | N | N | Y | N |
| renameTable | Y | Y | Y | Y | N |
| tableExists | Y | Y | Y | Y | N |
View Spec V1🔗
| View Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| createView | Y | N | N | Y | N |
| dropView | Y | N | N | Y | N |
| listView | Y | N | N | Y | N |
| viewExists | Y | N | N | Y | N |
| replaceView | Y | N | N | N | N |
| renameView | Y | N | N | N | N |
Namespace Operations🔗
| Namespace Operation | Java | PyIceberg | Rust | Go | C++ |
|---|---|---|---|---|---|
| listNamespaces | Y | Y | Y | Y | N |
| createNamespace | Y | Y | Y | Y | N |
| dropNamespace | Y | Y | Y | Y | N |
| namespaceExists | Y | Y | Y | Y | N |
| updateNamespaceProperties | Y | Y | Y | Y | N |
| loadNamespaceMetadata | Y | Y | Y | Y | N |