org.apache.iceberg.nessie

Class NessieCatalog

java.lang.Object

org.apache.iceberg.BaseMetastoreCatalog

org.apache.iceberg.nessie.NessieCatalog

All Implemented Interfaces:

java.lang.AutoCloseable, org.apache.hadoop.conf.Configurable, Catalog, SupportsNamespaces

public class NessieCatalog
extends BaseMetastoreCatalog
implements java.lang.AutoCloseable, SupportsNamespaces, org.apache.hadoop.conf.Configurable

Nessie implementation of Iceberg Catalog.

A note on namespaces: Nessie namespaces are implicit and do not need to be explicitly created or deleted. The create and delete namespace methods are no-ops for the NessieCatalog. One can still list namespaces that have objects stored in them to assist with namespace-centric catalog exploration.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.iceberg.BaseMetastoreCatalog

BaseMetastoreCatalog.BaseMetastoreCatalogTableBuilder

Nested classes/interfaces inherited from interface org.apache.iceberg.catalog.Catalog

Catalog.TableBuilder

Constructor Summary

Constructors

Constructor and Description
NessieCatalog()

Method Summary

Modifier and Type	Method and Description
void	close()
void	createNamespace(Namespace namespace, java.util.Map<java.lang.String,java.lang.String> metadata) creating namespaces in nessie is implicit, therefore this is a no-op.
java.lang.String	currentHash()
protected java.lang.String	defaultWarehouseLocation(TableIdentifier table)
boolean	dropNamespace(Namespace namespace) Namespaces in Nessie are implicit and deleting them results in a no-op.
boolean	dropTable(TableIdentifier identifier, boolean purge) Drop a table; optionally delete data and metadata files.
void	dropTableInner(TableIdentifier identifier)
org.apache.hadoop.conf.Configuration	getConf()
void	initialize(java.lang.String inputName, java.util.Map<java.lang.String,java.lang.String> options) Initialize a catalog given a custom name and a map of catalog properties.
java.util.List<Namespace>	listNamespaces(Namespace namespace) List namespaces from the namespace.
java.util.List<TableIdentifier>	listTables(Namespace namespace) Return all the identifiers under this namespace.
java.util.Map<java.lang.String,java.lang.String>	loadNamespaceMetadata(Namespace namespace) namespace metadata is not supported in Nessie and we return an empty map.
java.lang.String	name() Return the name for this catalog.
protected TableOperations	newTableOps(TableIdentifier tableIdentifier)
void	refresh()
boolean	removeProperties(Namespace namespace, java.util.Set<java.lang.String> properties) Remove a set of metadata from a namespace in the catalog.
void	renameTable(TableIdentifier from, TableIdentifier toOriginal) Rename a table.
void	setConf(org.apache.hadoop.conf.Configuration conf)
boolean	setProperties(Namespace namespace, java.util.Map<java.lang.String,java.lang.String> properties) Apply a set of metadata to a namespace in the catalog.

Methods inherited from class org.apache.iceberg.BaseMetastoreCatalog

buildTable, createTable, fullTableName, isValidIdentifier, loadTable, newCreateTableTransaction, newReplaceTableTransaction, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.iceberg.catalog.SupportsNamespaces

createNamespace, listNamespaces, namespaceExists

Methods inherited from interface org.apache.iceberg.catalog.Catalog

createTable, createTable, createTable, dropTable, newCreateTableTransaction, newCreateTableTransaction, newCreateTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction, tableExists

Constructor Detail

NessieCatalog

public NessieCatalog()

Method Detail

initialize

public void initialize(java.lang.String inputName,
                       java.util.Map<java.lang.String,java.lang.String> options)

Description copied from interface: Catalog

Initialize a catalog given a custom name and a map of catalog properties.

A custom Catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.

Specified by:

initialize in interface Catalog

Parameters:

inputName - a custom name for the catalog

options - catalog properties

close

public void close()

Specified by:

close in interface java.lang.AutoCloseable

name

public java.lang.String name()

Description copied from interface: Catalog

Return the name for this catalog.

Specified by:

name in interface Catalog

Returns:

this catalog's name

newTableOps

protected TableOperations newTableOps(TableIdentifier tableIdentifier)

Specified by:

newTableOps in class BaseMetastoreCatalog

defaultWarehouseLocation

protected java.lang.String defaultWarehouseLocation(TableIdentifier table)

Specified by:

defaultWarehouseLocation in class BaseMetastoreCatalog

listTables

public java.util.List<TableIdentifier> listTables(Namespace namespace)

Description copied from interface: Catalog

Return all the identifiers under this namespace.

Specified by:

listTables in interface Catalog

Parameters:

namespace - a namespace

Returns:

a list of identifiers for tables

dropTable

public boolean dropTable(TableIdentifier identifier,
                         boolean purge)

Description copied from interface: Catalog

Drop a table; optionally delete data and metadata files.

If purge is set to true the implementation should delete all data and metadata files.

Specified by:

dropTable in interface Catalog

Parameters:

identifier - a table identifier

purge - if true, delete all data and metadata files in the table

Returns:

true if the table was dropped, false if the table did not exist

renameTable

public void renameTable(TableIdentifier from,
                        TableIdentifier toOriginal)

Description copied from interface: Catalog

Rename a table.

Specified by:

renameTable in interface Catalog

Parameters:

from - identifier of the table to rename

toOriginal - new table name

createNamespace

public void createNamespace(Namespace namespace,
                            java.util.Map<java.lang.String,java.lang.String> metadata)

creating namespaces in nessie is implicit, therefore this is a no-op. Metadata is ignored.

Specified by:

createNamespace in interface SupportsNamespaces

Parameters:

namespace - a multi-part namespace

metadata - a string Map of properties for the given namespace

listNamespaces

public java.util.List<Namespace> listNamespaces(Namespace namespace)
                                         throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

List namespaces from the namespace.

For example, if table a.b.t exists, use 'SELECT NAMESPACE IN a' this method must return Namepace.of("a","b") Namespace.

Specified by:

listNamespaces in interface SupportsNamespaces

Returns:

a List of namespace Namespace names

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

loadNamespaceMetadata

public java.util.Map<java.lang.String,java.lang.String> loadNamespaceMetadata(Namespace namespace)
                                                                       throws NoSuchNamespaceException

namespace metadata is not supported in Nessie and we return an empty map.

Specified by:

loadNamespaceMetadata in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

Returns:

an empty map

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

dropNamespace

public boolean dropNamespace(Namespace namespace)
                      throws NamespaceNotEmptyException

Namespaces in Nessie are implicit and deleting them results in a no-op.

Specified by:

dropNamespace in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

Returns:

always false.

Throws:

NamespaceNotEmptyException - If the namespace does not empty

setProperties

public boolean setProperties(Namespace namespace,
                             java.util.Map<java.lang.String,java.lang.String> properties)
                      throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

Apply a set of metadata to a namespace in the catalog.

Specified by:

setProperties in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

properties - a collection of metadata to apply to the namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

removeProperties

public boolean removeProperties(Namespace namespace,
                                java.util.Set<java.lang.String> properties)
                         throws NoSuchNamespaceException

Description copied from interface: SupportsNamespaces

Remove a set of metadata from a namespace in the catalog.

Specified by:

removeProperties in interface SupportsNamespaces

Parameters:

namespace - a namespace. Namespace

properties - a collection of metadata to apply to the namespace

Throws:

NoSuchNamespaceException - If the namespace does not exist (optional)

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

refresh

public void refresh()
             throws com.dremio.nessie.error.NessieNotFoundException

Throws:

com.dremio.nessie.error.NessieNotFoundException

currentHash

public java.lang.String currentHash()

dropTableInner

public void dropTableInner(TableIdentifier identifier)
                    throws com.dremio.nessie.error.NessieConflictException,
                           com.dremio.nessie.error.NessieNotFoundException

Throws:

com.dremio.nessie.error.NessieConflictException

com.dremio.nessie.error.NessieNotFoundException