org.apache.iceberg.hadoop

Class HadoopTables

java.lang.Object

org.apache.iceberg.hadoop.HadoopTables

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, Tables

public class HadoopTables
extends java.lang.Object
implements Tables, org.apache.hadoop.conf.Configurable

Implementation of Iceberg tables that uses the Hadoop FileSystem to store metadata and manifests.

Constructor Summary

Constructors

Constructor and Description
HadoopTables()
HadoopTables(org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method and Description
Catalog.TableBuilder	buildTable(java.lang.String location, Schema schema)
Table	create(Schema schema, PartitionSpec spec, SortOrder order, java.util.Map<java.lang.String,java.lang.String> properties, java.lang.String location) Create a table using the FileSystem implementation resolve from location.
boolean	dropTable(java.lang.String location) Drop a table and delete all data and metadata files.
boolean	dropTable(java.lang.String location, boolean purge) Drop a table; optionally delete data and metadata files.
boolean	exists(java.lang.String location)
org.apache.hadoop.conf.Configuration	getConf()
Table	load(java.lang.String location) Loads the table location from a FileSystem path location.
Transaction	newCreateTableTransaction(java.lang.String location, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> properties) Start a transaction to create a table.
Transaction	newReplaceTableTransaction(java.lang.String location, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> properties, boolean orCreate) Start a transaction to replace a table.
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.Tables

create, create, create

Constructor Detail

HadoopTables

public HadoopTables()

HadoopTables

public HadoopTables(org.apache.hadoop.conf.Configuration conf)

Method Detail

load

public Table load(java.lang.String location)

Loads the table location from a FileSystem path location.

Specified by:

load in interface Tables

Parameters:

location - a path URI (e.g. hdfs:///warehouse/my_table/)

Returns:

table implementation

exists

public boolean exists(java.lang.String location)

Specified by:

exists in interface Tables

create

public Table create(Schema schema,
                    PartitionSpec spec,
                    SortOrder order,
                    java.util.Map<java.lang.String,java.lang.String> properties,
                    java.lang.String location)

Create a table using the FileSystem implementation resolve from location.

Specified by:

create in interface Tables

Parameters:

schema - iceberg schema used to create the table

spec - partitioning spec, if null the table will be unpartitioned

properties - a string map of table properties, initialized to empty if null

location - a path URI (e.g. hdfs:///warehouse/my_table)

Returns:

newly created table implementation

dropTable

public boolean dropTable(java.lang.String location)

Drop a table and delete all data and metadata files.

Parameters:

location - a path URI (e.g. hdfs:///warehouse/my_table)

Returns:

true if the table was dropped, false if it did not exist

dropTable

public boolean dropTable(java.lang.String location,
                         boolean purge)

Drop a table; optionally delete data and metadata files.

If purge is set to true the implementation should delete all data and metadata files.

Parameters:

location - a path URI (e.g. hdfs:///warehouse/my_table)

purge - if true, delete all data and metadata files in the table

Returns:

true if the table was dropped, false if it did not exist

newCreateTableTransaction

public Transaction newCreateTableTransaction(java.lang.String location,
                                             Schema schema,
                                             PartitionSpec spec,
                                             java.util.Map<java.lang.String,java.lang.String> properties)

Start a transaction to create a table.

Parameters:

location - a location for the table

schema - a schema

spec - a partition spec

properties - a string map of table properties

Returns:

a Transaction to create the table

Throws:

AlreadyExistsException - if the table already exists

newReplaceTableTransaction

public Transaction newReplaceTableTransaction(java.lang.String location,
                                              Schema schema,
                                              PartitionSpec spec,
                                              java.util.Map<java.lang.String,java.lang.String> properties,
                                              boolean orCreate)

Start a transaction to replace a table.

Parameters:

location - a location for the table

schema - a schema

spec - a partition spec

properties - a string map of table properties

orCreate - whether to create the table if not exists

Returns:

a Transaction to replace the table

Throws:

NoSuchTableException - if the table doesn't exist and orCreate is false

buildTable

public Catalog.TableBuilder buildTable(java.lang.String location,
                                       Schema schema)

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable