org.apache.iceberg.mr

Class Catalogs

java.lang.Object

org.apache.iceberg.mr.Catalogs

public final class Catalogs
extends java.lang.Object

Class for catalog resolution and accessing the common functions for Catalog API.

If the catalog name is provided, get the catalog type from iceberg.catalog.catalogName.type config.

In case the catalog name is location_based_table, type is ignored and tables will be loaded using HadoopTables.

In case the value of catalog type is null, iceberg.catalog.catalogName.catalog-impl config is used to determine the catalog implementation class.

If catalog name is null, get the catalog type from iceberg.mr.catalog config:

• hive: HiveCatalog
• location: HadoopTables
• hadoop: HadoopCatalog

In case the value of catalog type is null, iceberg.mr.catalog.loader.class is used to determine the catalog implementation class.

Note: null catalog name mode is only supported for backwards compatibility. Using this mode is NOT RECOMMENDED.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	ICEBERG_DEFAULT_CATALOG_NAME
static java.lang.String	ICEBERG_HADOOP_TABLE_NAME
static java.lang.String	LOCATION
static java.lang.String	NAME

Method Summary

Modifier and Type	Method and Description
static Table	createTable(org.apache.hadoop.conf.Configuration conf, java.util.Properties props) Creates an Iceberg table using the catalog specified by the configuration.
static boolean	dropTable(org.apache.hadoop.conf.Configuration conf, java.util.Properties props) Drops an Iceberg table using the catalog specified by the configuration.
static boolean	hiveCatalog(org.apache.hadoop.conf.Configuration conf, java.util.Properties props) Returns true if HiveCatalog is used
static Table	loadTable(org.apache.hadoop.conf.Configuration conf) Load an Iceberg table using the catalog and table identifier (or table path) specified by the configuration.
static Table	loadTable(org.apache.hadoop.conf.Configuration conf, java.util.Properties props) Load an Iceberg table using the catalog specified by the configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ICEBERG_DEFAULT_CATALOG_NAME

public static final java.lang.String ICEBERG_DEFAULT_CATALOG_NAME

See Also:

Constant Field Values

ICEBERG_HADOOP_TABLE_NAME

public static final java.lang.String ICEBERG_HADOOP_TABLE_NAME

See Also:

Constant Field Values

NAME

public static final java.lang.String NAME

See Also:

Constant Field Values

LOCATION

public static final java.lang.String LOCATION

See Also:

Constant Field Values

Method Detail

loadTable

public static Table loadTable(org.apache.hadoop.conf.Configuration conf)

Load an Iceberg table using the catalog and table identifier (or table path) specified by the configuration.

Parameters:

conf - a Hadoop conf

Returns:

an Iceberg table

loadTable

public static Table loadTable(org.apache.hadoop.conf.Configuration conf,
                              java.util.Properties props)

Load an Iceberg table using the catalog specified by the configuration.

The table identifier (NAME) and the catalog name (InputFormatConfig.CATALOG_NAME), or table path (LOCATION) should be specified by the controlling properties.

Used by HiveIcebergSerDe and HiveIcebergStorageHandler

Parameters:

conf - a Hadoop

props - the controlling properties

Returns:

an Iceberg table

createTable

public static Table createTable(org.apache.hadoop.conf.Configuration conf,
                                java.util.Properties props)

Creates an Iceberg table using the catalog specified by the configuration.

The properties should contain the following values:

• Table identifier (NAME) or table path (LOCATION) is required
• Table schema (InputFormatConfig.TABLE_SCHEMA) is required
• Partition specification (InputFormatConfig.PARTITION_SPEC) is optional. Table will be unpartitioned if not provided

Other properties will be handled over to the Table creation. The controlling properties above will not be propagated.

Parameters:

conf - a Hadoop conf

props - the controlling properties

Returns:

the created Iceberg table

dropTable

public static boolean dropTable(org.apache.hadoop.conf.Configuration conf,
                                java.util.Properties props)

Drops an Iceberg table using the catalog specified by the configuration.

The table identifier (NAME) or table path (LOCATION) should be specified by the controlling properties.

Parameters:

conf - a Hadoop conf

props - the controlling properties

Returns:

the created Iceberg table

hiveCatalog

public static boolean hiveCatalog(org.apache.hadoop.conf.Configuration conf,
                                  java.util.Properties props)

Returns true if HiveCatalog is used

Parameters:

conf - a Hadoop conf

props - the controlling properties

Returns:

true if the Catalog is HiveCatalog