org.apache.iceberg.jdbc

Class JdbcCatalog

java.lang.Object

org.apache.iceberg.BaseMetastoreCatalog

org.apache.iceberg.view.BaseMetastoreViewCatalog

org.apache.iceberg.jdbc.JdbcCatalog

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, Catalog, SupportsNamespaces, ViewCatalog, Configurable<java.lang.Object>

public class JdbcCatalog
extends BaseMetastoreViewCatalog
implements Configurable<java.lang.Object>, SupportsNamespaces

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	JdbcCatalog.ViewAwareTableBuilder The purpose of this class is to add view detection only when SchemaVersion.V1 schema is used when replacing a table.

Nested classes/interfaces inherited from class org.apache.iceberg.view.BaseMetastoreViewCatalog

BaseMetastoreViewCatalog.BaseMetastoreViewCatalogTableBuilder, BaseMetastoreViewCatalog.BaseViewBuilder

Nested classes/interfaces inherited from class org.apache.iceberg.BaseMetastoreCatalog

BaseMetastoreCatalog.BaseMetastoreCatalogTableBuilder

Nested classes/interfaces inherited from interface org.apache.iceberg.catalog.Catalog

Catalog.TableBuilder

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	PROPERTY_PREFIX

Constructor Summary

Constructors

Constructor and Description
JdbcCatalog()
JdbcCatalog(java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,FileIO> ioBuilder, java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,JdbcClientPool> clientPoolBuilder, boolean initializeCatalogTables)

Method Summary

Modifier and Type	Method and Description
Catalog.TableBuilder	buildTable(TableIdentifier identifier, Schema schema) /** Instantiate a builder to either create a table or start a create/replace transaction.
void	close()
void	createNamespace(Namespace namespace, java.util.Map<java.lang.String,java.lang.String> metadata) Create a namespace in the catalog.
protected java.lang.String	defaultWarehouseLocation(TableIdentifier table)
boolean	dropNamespace(Namespace namespace) Drop a namespace.
boolean	dropTable(TableIdentifier identifier, boolean purge) Drop a table; optionally delete data and metadata files.
boolean	dropView(TableIdentifier identifier) Drop a view.
void	initialize(java.lang.String name, java.util.Map<java.lang.String,java.lang.String> properties) Initialize a view catalog given a custom name and a map of catalog properties.
java.util.List<Namespace>	listNamespaces() List top-level namespaces from the catalog.
java.util.List<Namespace>	listNamespaces(Namespace namespace) List child namespaces from the namespace.
java.util.List<TableIdentifier>	listTables(Namespace namespace) Return all the identifiers under this namespace.
java.util.List<TableIdentifier>	listViews(Namespace namespace) Return all the identifiers under this namespace.
java.util.Map<java.lang.String,java.lang.String>	loadNamespaceMetadata(Namespace namespace) Load metadata properties for a namespace.
java.lang.String	name() Return the name for this catalog.
boolean	namespaceExists(Namespace namespace) Checks whether the Namespace exists.
protected TableOperations	newTableOps(TableIdentifier tableIdentifier)
protected ViewOperations	newViewOps(TableIdentifier viewIdentifier)
protected java.util.Map<java.lang.String,java.lang.String>	properties()
boolean	removeProperties(Namespace namespace, java.util.Set<java.lang.String> properties) Remove a set of property keys from a namespace in the catalog.
void	renameTable(TableIdentifier from, TableIdentifier to) Rename a table.
void	renameView(TableIdentifier from, TableIdentifier to) Rename a view.
void	setConf(java.lang.Object conf)
boolean	setProperties(Namespace namespace, java.util.Map<java.lang.String,java.lang.String> properties) Set a collection of properties on a namespace in the catalog.

Methods inherited from class org.apache.iceberg.view.BaseMetastoreViewCatalog

buildView, loadView

Methods inherited from class org.apache.iceberg.BaseMetastoreCatalog

fullTableName, isValidIdentifier, loadTable, metricsReporter, registerTable, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.iceberg.catalog.SupportsNamespaces

createNamespace

Methods inherited from interface org.apache.iceberg.catalog.ViewCatalog

invalidateView, viewExists

Methods inherited from interface org.apache.iceberg.catalog.Catalog

createTable, createTable, createTable, createTable, dropTable, invalidateTable, newCreateTableTransaction, newCreateTableTransaction, newCreateTableTransaction, newCreateTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction, tableExists

Field Detail

PROPERTY_PREFIX

public static final java.lang.String PROPERTY_PREFIX

See Also:

Constant Field Values

Constructor Detail

JdbcCatalog

public JdbcCatalog()

JdbcCatalog

public JdbcCatalog(java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,FileIO> ioBuilder,
                   java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,JdbcClientPool> clientPoolBuilder,
                   boolean initializeCatalogTables)

Method Detail

initialize

public void initialize(java.lang.String name,
                       java.util.Map<java.lang.String,java.lang.String> properties)

Description copied from interface: ViewCatalog

Initialize a view catalog given a custom name and a map of catalog properties.

A custom view catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.

Specified by:

initialize in interface Catalog

Specified by:

initialize in interface ViewCatalog

Overrides:

initialize in class BaseMetastoreViewCatalog

Parameters:

name - a custom name for the catalog

properties - catalog properties

newTableOps

protected TableOperations newTableOps(TableIdentifier tableIdentifier)

Specified by:

newTableOps in class BaseMetastoreCatalog

newViewOps

protected ViewOperations newViewOps(TableIdentifier viewIdentifier)

Specified by:

newViewOps in class BaseMetastoreViewCatalog

defaultWarehouseLocation

protected java.lang.String defaultWarehouseLocation(TableIdentifier table)

Specified by:

defaultWarehouseLocation in class BaseMetastoreCatalog

dropTable

public boolean dropTable(TableIdentifier identifier,
                         boolean purge)

Description copied from interface: Catalog

Drop a table; optionally delete data and metadata files.

If purge is set to true the implementation should delete all data and metadata files.

Specified by:

dropTable in interface Catalog

Parameters:

identifier - a table identifier

purge - if true, delete all data and metadata files in the table

Returns:

true if the table was dropped, false if the table did not exist

listTables

public java.util.List<TableIdentifier> listTables(Namespace namespace)

Description copied from interface: Catalog

Return all the identifiers under this namespace.

Specified by:

listTables in interface Catalog

Parameters:

namespace - a namespace

Returns:

a list of identifiers for tables

renameTable

public void renameTable(TableIdentifier from,
                        TableIdentifier to)

Description copied from interface: Catalog

Rename a table.

Specified by:

renameTable in interface Catalog

Parameters:

from - identifier of the table to rename

to - new table name

name

public java.lang.String name()

Description copied from interface: ViewCatalog

Return the name for this catalog.

Specified by:

name in interface Catalog

Specified by:

name in interface ViewCatalog

Overrides:

name in class BaseMetastoreViewCatalog

Returns:

this catalog's name

setConf

public void setConf(java.lang.Object conf)

Specified by:

setConf in interface Configurable<java.lang.Object>

createNamespace

public void createNamespace(Namespace namespace,
                            java.util.Map<java.lang.String,java.lang.String> metadata)

Description copied from interface: SupportsNamespaces

Create a namespace in the catalog.

Specified by:

createNamespace in interface SupportsNamespaces

Parameters:

namespace - a multi-part namespace

metadata - a string Map of properties for the given namespace

listNamespaces

public java.util.List<Namespace> listNamespaces()

Description copied from interface: SupportsNamespaces

List top-level namespaces from the catalog.

If an object such as a table, view, or function exists, its parent namespaces must also exist and must be returned by this discovery method. For example, if table a.b.t exists, this method must return ["a"] in the result array.

Specified by:

listNamespaces in interface SupportsNamespaces

Returns:

a List of namespace Namespace names

listNamespaces

public java.util.List<Namespace> listNamespaces(Namespace namespace)
                                         throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

List child namespaces from the namespace.

For two existing tables named 'a.b.c.table' and 'a.b.d.table', this method returns:

• Given: Namespace.empty()
• Returns: Namespace.of("a")

• Given: Namespace.of("a")
• Returns: Namespace.of("a", "b")

• Given: Namespace.of("a", "b")
• Returns: Namespace.of("a", "b", "c") and Namespace.of("a", "b", "d")

• Given: Namespace.of("a", "b", "c")
• Returns: empty list, because there are no child namespaces

Specified by:

listNamespaces in interface SupportsNamespaces

Returns:

a List of child Namespace names from the given namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

loadNamespaceMetadata

public java.util.Map<java.lang.String,java.lang.String> loadNamespaceMetadata(Namespace namespace)
                                                                       throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

Load metadata properties for a namespace.

Specified by:

loadNamespaceMetadata in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

Returns:

a string map of properties for the given namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

dropNamespace

public boolean dropNamespace(Namespace namespace)
                      throws NamespaceNotEmptyException

Description copied from interface: SupportsNamespaces

Drop a namespace. If the namespace exists and was dropped, this will return true.

Specified by:

dropNamespace in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

Returns:

true if the namespace was dropped, false otherwise.

Throws:

NamespaceNotEmptyException - If the namespace is not empty

setProperties

public boolean setProperties(Namespace namespace,
                             java.util.Map<java.lang.String,java.lang.String> properties)
                      throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

Set a collection of properties on a namespace in the catalog.

Properties that are not in the given map are not modified or removed by this method.

Specified by:

setProperties in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

properties - a collection of metadata to apply to the namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

removeProperties

public boolean removeProperties(Namespace namespace,
                                java.util.Set<java.lang.String> properties)
                         throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

Remove a set of property keys from a namespace in the catalog.

Properties that are not in the given set are not modified or removed by this method.

Specified by:

removeProperties in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

properties - a collection of metadata to apply to the namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

close

public void close()

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Overrides:

close in class BaseMetastoreCatalog

namespaceExists

public boolean namespaceExists(Namespace namespace)

Description copied from interface: SupportsNamespaces

Checks whether the Namespace exists.

Specified by:

namespaceExists in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

Returns:

true if the Namespace exists, false otherwise

dropView

public boolean dropView(TableIdentifier identifier)

Description copied from interface: ViewCatalog

Drop a view.

Specified by:

dropView in interface ViewCatalog

Parameters:

identifier - a view identifier

Returns:

true if the view was dropped, false if the view did not exist

listViews

public java.util.List<TableIdentifier> listViews(Namespace namespace)

Description copied from interface: ViewCatalog

Return all the identifiers under this namespace.

Specified by:

listViews in interface ViewCatalog

Parameters:

namespace - a namespace

Returns:

a list of identifiers for views

renameView

public void renameView(TableIdentifier from,
                       TableIdentifier to)

Description copied from interface: ViewCatalog

Rename a view.

Specified by:

renameView in interface ViewCatalog

Parameters:

from - identifier of the view to rename

to - new view identifier

properties

protected java.util.Map<java.lang.String,java.lang.String> properties()

Overrides:

properties in class BaseMetastoreCatalog

buildTable

public Catalog.TableBuilder buildTable(TableIdentifier identifier,
                                       Schema schema)

Description copied from interface: Catalog

/** Instantiate a builder to either create a table or start a create/replace transaction.

Specified by:

buildTable in interface Catalog

Overrides:

buildTable in class BaseMetastoreViewCatalog

Parameters:

identifier - a table identifier

schema - a schema

Returns:

the builder to create a table or start a create/replace transaction