org.apache.iceberg.rest

Class RESTCatalog

java.lang.Object

org.apache.iceberg.rest.RESTCatalog

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, Catalog, SupportsNamespaces, Configurable<org.apache.hadoop.conf.Configuration>

public class RESTCatalog
extends java.lang.Object
implements Catalog, SupportsNamespaces, Configurable<org.apache.hadoop.conf.Configuration>, java.io.Closeable

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.iceberg.catalog.Catalog

Catalog.TableBuilder

Constructor Summary

Constructors

Constructor and Description
RESTCatalog()
RESTCatalog(java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,RESTClient> clientBuilder)
RESTCatalog(SessionCatalog.SessionContext context, java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,RESTClient> clientBuilder)

Method Summary

Modifier and Type	Method and Description
Catalog.TableBuilder	buildTable(TableIdentifier ident, Schema schema) /** Instantiate a builder to either create a table or start a create/replace transaction.
void	close()
void	createNamespace(Namespace ns, java.util.Map<java.lang.String,java.lang.String> props) Create a namespace in the catalog.
Table	createTable(TableIdentifier identifier, Schema schema) Create an unpartitioned table.
Table	createTable(TableIdentifier ident, Schema schema, PartitionSpec spec) Create a table.
Table	createTable(TableIdentifier ident, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> props) Create a table.
Table	createTable(TableIdentifier ident, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> props) Create a table.
boolean	dropNamespace(Namespace ns) Drop a namespace.
boolean	dropTable(TableIdentifier ident) Drop a table and delete all data and metadata files.
boolean	dropTable(TableIdentifier ident, boolean purge) Drop a table; optionally delete data and metadata files.
void	initialize(java.lang.String name, java.util.Map<java.lang.String,java.lang.String> props) Initialize a catalog given a custom name and a map of catalog properties.
void	invalidateTable(TableIdentifier ident) Invalidate cached table metadata from current catalog.
java.util.List<Namespace>	listNamespaces(Namespace ns) List namespaces from the namespace.
java.util.List<TableIdentifier>	listTables(Namespace ns) Return all the identifiers under this namespace.
java.util.Map<java.lang.String,java.lang.String>	loadNamespaceMetadata(Namespace ns) Load metadata properties for a namespace.
Table	loadTable(TableIdentifier ident) Load a table.
java.lang.String	name() Return the name for this catalog.
Transaction	newCreateTableTransaction(TableIdentifier identifier, Schema schema) Start a transaction to create a table.
Transaction	newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec) Start a transaction to create a table.
Transaction	newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> props) Start a transaction to create a table.
Transaction	newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> props) Start a transaction to create a table.
Transaction	newReplaceTableTransaction(TableIdentifier ident, Schema schema, boolean orCreate) Start a transaction to replace a table.
Transaction	newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, boolean orCreate) Start a transaction to replace a table.
Transaction	newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> props, boolean orCreate) Start a transaction to replace a table.
Transaction	newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> props, boolean orCreate) Start a transaction to replace a table.
java.util.Map<java.lang.String,java.lang.String>	properties()
Table	registerTable(TableIdentifier ident, java.lang.String metadataFileLocation) Register a table with the catalog if it does not exist.
boolean	removeProperties(Namespace ns, java.util.Set<java.lang.String> props) Remove a set of property keys from a namespace in the catalog.
void	renameTable(TableIdentifier from, TableIdentifier to) Rename a table.
void	setConf(org.apache.hadoop.conf.Configuration conf)
boolean	setProperties(Namespace ns, java.util.Map<java.lang.String,java.lang.String> props) Set a collection of properties on a namespace in the catalog.
boolean	tableExists(TableIdentifier ident) Check whether table exists.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.catalog.SupportsNamespaces

createNamespace, listNamespaces, namespaceExists

Constructor Detail

RESTCatalog

public RESTCatalog()

RESTCatalog

public RESTCatalog(java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,RESTClient> clientBuilder)

RESTCatalog

public RESTCatalog(SessionCatalog.SessionContext context,
                   java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,RESTClient> clientBuilder)

Method Detail

initialize

public void initialize(java.lang.String name,
                       java.util.Map<java.lang.String,java.lang.String> props)

Description copied from interface: Catalog

Initialize a catalog given a custom name and a map of catalog properties.

A custom Catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.

Specified by:

initialize in interface Catalog

Parameters:

name - a custom name for the catalog

props - catalog properties

name

public java.lang.String name()

Description copied from interface: Catalog

Return the name for this catalog.

Specified by:

name in interface Catalog

Returns:

this catalog's name

properties

public java.util.Map<java.lang.String,java.lang.String> properties()

listTables

public java.util.List<TableIdentifier> listTables(Namespace ns)

Description copied from interface: Catalog

Return all the identifiers under this namespace.

Specified by:

listTables in interface Catalog

Parameters:

ns - a namespace

Returns:

a list of identifiers for tables

tableExists

public boolean tableExists(TableIdentifier ident)

Description copied from interface: Catalog

Check whether table exists.

Specified by:

tableExists in interface Catalog

Parameters:

ident - a table identifier

Returns:

true if the table exists, false otherwise

loadTable

public Table loadTable(TableIdentifier ident)

Description copied from interface: Catalog

Load a table.

Specified by:

loadTable in interface Catalog

Parameters:

ident - a table identifier

Returns:

instance of Table implementation referred by tableIdentifier

invalidateTable

public void invalidateTable(TableIdentifier ident)

Description copied from interface: Catalog

Invalidate cached table metadata from current catalog.

If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing.

Specified by:

invalidateTable in interface Catalog

Parameters:

ident - a table identifier

buildTable

public Catalog.TableBuilder buildTable(TableIdentifier ident,
                                       Schema schema)

Description copied from interface: Catalog

/** Instantiate a builder to either create a table or start a create/replace transaction.

Specified by:

buildTable in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

Returns:

the builder to create a table or start a create/replace transaction

createTable

public Table createTable(TableIdentifier ident,
                         Schema schema,
                         PartitionSpec spec,
                         java.lang.String location,
                         java.util.Map<java.lang.String,java.lang.String> props)

Description copied from interface: Catalog

Create a table.

Specified by:

createTable in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

spec - a partition spec

location - a location for the table; leave null if unspecified

props - a string map of table properties

Returns:

a Table instance

createTable

public Table createTable(TableIdentifier ident,
                         Schema schema,
                         PartitionSpec spec,
                         java.util.Map<java.lang.String,java.lang.String> props)

Description copied from interface: Catalog

Create a table.

Specified by:

createTable in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

spec - a partition spec

props - a string map of table properties

Returns:

a Table instance

createTable

public Table createTable(TableIdentifier ident,
                         Schema schema,
                         PartitionSpec spec)

Description copied from interface: Catalog

Create a table.

Specified by:

createTable in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

spec - a partition spec

Returns:

a Table instance

createTable

public Table createTable(TableIdentifier identifier,
                         Schema schema)

Description copied from interface: Catalog

Create an unpartitioned table.

Specified by:

createTable in interface Catalog

Parameters:

identifier - a table identifier

schema - a schema

Returns:

a Table instance

newCreateTableTransaction

public Transaction newCreateTableTransaction(TableIdentifier ident,
                                             Schema schema,
                                             PartitionSpec spec,
                                             java.lang.String location,
                                             java.util.Map<java.lang.String,java.lang.String> props)

Description copied from interface: Catalog

Start a transaction to create a table.

Specified by:

newCreateTableTransaction in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

spec - a partition spec

location - a location for the table; leave null if unspecified

props - a string map of table properties

Returns:

a Transaction to create the table

newCreateTableTransaction

public Transaction newCreateTableTransaction(TableIdentifier ident,
                                             Schema schema,
                                             PartitionSpec spec,
                                             java.util.Map<java.lang.String,java.lang.String> props)

Description copied from interface: Catalog

Start a transaction to create a table.

Specified by:

newCreateTableTransaction in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

spec - a partition spec

props - a string map of table properties

Returns:

a Transaction to create the table

newCreateTableTransaction

public Transaction newCreateTableTransaction(TableIdentifier ident,
                                             Schema schema,
                                             PartitionSpec spec)

Description copied from interface: Catalog

Start a transaction to create a table.

Specified by:

newCreateTableTransaction in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

spec - a partition spec

Returns:

a Transaction to create the table

newCreateTableTransaction

public Transaction newCreateTableTransaction(TableIdentifier identifier,
                                             Schema schema)

Description copied from interface: Catalog

Start a transaction to create a table.

Specified by:

newCreateTableTransaction in interface Catalog

Parameters:

identifier - a table identifier

schema - a schema

Returns:

a Transaction to create the table

newReplaceTableTransaction

public Transaction newReplaceTableTransaction(TableIdentifier ident,
                                              Schema schema,
                                              PartitionSpec spec,
                                              java.lang.String location,
                                              java.util.Map<java.lang.String,java.lang.String> props,
                                              boolean orCreate)

Description copied from interface: Catalog

Start a transaction to replace a table.

Specified by:

newReplaceTableTransaction in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

spec - a partition spec

location - a location for the table; leave null if unspecified

props - a string map of table properties

orCreate - whether to create the table if not exists

Returns:

a Transaction to replace the table

newReplaceTableTransaction

public Transaction newReplaceTableTransaction(TableIdentifier ident,
                                              Schema schema,
                                              PartitionSpec spec,
                                              java.util.Map<java.lang.String,java.lang.String> props,
                                              boolean orCreate)

Description copied from interface: Catalog

Start a transaction to replace a table.

Specified by:

newReplaceTableTransaction in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

spec - a partition spec

props - a string map of table properties

orCreate - whether to create the table if not exists

Returns:

a Transaction to replace the table

newReplaceTableTransaction

public Transaction newReplaceTableTransaction(TableIdentifier ident,
                                              Schema schema,
                                              PartitionSpec spec,
                                              boolean orCreate)

Description copied from interface: Catalog

Start a transaction to replace a table.

Specified by:

newReplaceTableTransaction in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

spec - a partition spec

orCreate - whether to create the table if not exists

Returns:

a Transaction to replace the table

newReplaceTableTransaction

public Transaction newReplaceTableTransaction(TableIdentifier ident,
                                              Schema schema,
                                              boolean orCreate)

Description copied from interface: Catalog

Start a transaction to replace a table.

Specified by:

newReplaceTableTransaction in interface Catalog

Parameters:

ident - a table identifier

schema - a schema

orCreate - whether to create the table if not exists

Returns:

a Transaction to replace the table

dropTable

public boolean dropTable(TableIdentifier ident)

Description copied from interface: Catalog

Drop a table and delete all data and metadata files.

Specified by:

dropTable in interface Catalog

Parameters:

ident - a table identifier

Returns:

true if the table was dropped, false if the table did not exist

dropTable

public boolean dropTable(TableIdentifier ident,
                         boolean purge)

Description copied from interface: Catalog

Drop a table; optionally delete data and metadata files.

If purge is set to true the implementation should delete all data and metadata files.

Specified by:

dropTable in interface Catalog

Parameters:

ident - a table identifier

purge - if true, delete all data and metadata files in the table

Returns:

true if the table was dropped, false if the table did not exist

renameTable

public void renameTable(TableIdentifier from,
                        TableIdentifier to)

Description copied from interface: Catalog

Rename a table.

Specified by:

renameTable in interface Catalog

Parameters:

from - identifier of the table to rename

to - new table name

registerTable

public Table registerTable(TableIdentifier ident,
                           java.lang.String metadataFileLocation)

Description copied from interface: Catalog

Register a table with the catalog if it does not exist.

Specified by:

registerTable in interface Catalog

Parameters:

ident - a table identifier

metadataFileLocation - the location of a metadata file

Returns:

a Table instance

createNamespace

public void createNamespace(Namespace ns,
                            java.util.Map<java.lang.String,java.lang.String> props)

Description copied from interface: SupportsNamespaces

Create a namespace in the catalog.

Specified by:

createNamespace in interface SupportsNamespaces

Parameters:

ns - a multi-part namespace

props - a string Map of properties for the given namespace

listNamespaces

public java.util.List<Namespace> listNamespaces(Namespace ns)
                                         throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

List namespaces from the namespace.

For example, if table a.b.t exists, use 'SELECT NAMESPACE IN a' this method must return Namepace.of("a","b") Namespace.

Specified by:

listNamespaces in interface SupportsNamespaces

Returns:

a List of namespace Namespace names

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

loadNamespaceMetadata

public java.util.Map<java.lang.String,java.lang.String> loadNamespaceMetadata(Namespace ns)
                                                                       throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

Load metadata properties for a namespace.

Specified by:

loadNamespaceMetadata in interface SupportsNamespaces

Parameters:

ns - a namespace. Namespace

Returns:

a string map of properties for the given namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

dropNamespace

public boolean dropNamespace(Namespace ns)
                      throws NamespaceNotEmptyException

Description copied from interface: SupportsNamespaces

Drop a namespace. If the namespace exists and was dropped, this will return true.

Specified by:

dropNamespace in interface SupportsNamespaces

Parameters:

ns - a namespace. Namespace

Returns:

true if the namespace was dropped, false otherwise.

Throws:

NamespaceNotEmptyException - If the namespace is not empty

setProperties

public boolean setProperties(Namespace ns,
                             java.util.Map<java.lang.String,java.lang.String> props)
                      throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

Set a collection of properties on a namespace in the catalog.

Properties that are not in the given map are not modified or removed by this method.

Specified by:

setProperties in interface SupportsNamespaces

Parameters:

ns - a namespace. Namespace

props - a collection of metadata to apply to the namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

removeProperties

public boolean removeProperties(Namespace ns,
                                java.util.Set<java.lang.String> props)
                         throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

Remove a set of property keys from a namespace in the catalog.

Properties that are not in the given set are not modified or removed by this method.

Specified by:

removeProperties in interface SupportsNamespaces

Parameters:

ns - a namespace. Namespace

props - a collection of metadata to apply to the namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface Configurable<org.apache.hadoop.conf.Configuration>

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Throws:

java.io.IOException