Package org.apache.iceberg.rest

Class RESTSessionCatalog

java.lang.Object

org.apache.iceberg.catalog.BaseSessionCatalog

org.apache.iceberg.catalog.BaseViewSessionCatalog

org.apache.iceberg.rest.RESTSessionCatalog

All Implemented Interfaces:

Closeable, AutoCloseable, SessionCatalog, ViewSessionCatalog, Configurable<Object>

public class RESTSessionCatalog
extends BaseViewSessionCatalog
implements Configurable<Object>, Closeable

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.iceberg.catalog.BaseViewSessionCatalog

BaseViewSessionCatalog.AsViewCatalog

Nested classes/interfaces inherited from class org.apache.iceberg.catalog.BaseSessionCatalog

BaseSessionCatalog.AsCatalog

Nested classes/interfaces inherited from interface org.apache.iceberg.catalog.SessionCatalog

SessionCatalog.SessionContext

Field Summary

Fields

Modifier and Type	Field	Description
static final String	REST_PAGE_SIZE	Deprecated. will be removed in 1.12.0.

Constructor Summary

Constructors

Constructor	Description
RESTSessionCatalog()
RESTSessionCatalog(Function<Map<String,String>,RESTClient> clientBuilder, BiFunction<SessionCatalog.SessionContext,Map<String,String>,FileIO> ioBuilder)

Method Summary

Modifier and Type	Method	Description
Catalog.TableBuilder	buildTable(SessionCatalog.SessionContext context, TableIdentifier identifier, Schema schema)	Create a builder to create a table or start a create/replace transaction.
ViewBuilder	buildView(SessionCatalog.SessionContext context, TableIdentifier identifier)	Instantiate a builder to create or replace a SQL view.
void	close()
void	commitTransaction(SessionCatalog.SessionContext context, List<TableCommit> commits)
void	createNamespace(SessionCatalog.SessionContext context, Namespace namespace, Map<String,String> metadata)	Create a namespace in the catalog.
protected org.apache.iceberg.rest.RESTTableCache	createTableCache(Map<String,String> props)
boolean	dropNamespace(SessionCatalog.SessionContext context, Namespace ns)	Drop a namespace.
boolean	dropTable(SessionCatalog.SessionContext context, TableIdentifier identifier)	Drop a table, without requesting that files are immediately deleted.
boolean	dropView(SessionCatalog.SessionContext context, TableIdentifier identifier)	Drop a view.
void	initialize(String name, Map<String,String> unresolved)	Initialize given a custom name and a map of catalog properties.
void	invalidateTable(SessionCatalog.SessionContext context, TableIdentifier ident)	Invalidate cached table metadata from current catalog.
List<Namespace>	listNamespaces(SessionCatalog.SessionContext context, Namespace namespace)	List child namespaces from the namespace.
List<TableIdentifier>	listTables(SessionCatalog.SessionContext context, Namespace ns)	Return all the identifiers under this namespace.
List<TableIdentifier>	listViews(SessionCatalog.SessionContext context, Namespace namespace)	Return all the identifiers under this namespace.
Map<String,String>	loadNamespaceMetadata(SessionCatalog.SessionContext context, Namespace ns)	Load metadata properties for a namespace.
Table	loadTable(SessionCatalog.SessionContext context, TableIdentifier identifier)	Load a table.
View	loadView(SessionCatalog.SessionContext context, TableIdentifier identifier)	Load a view.
boolean	namespaceExists(SessionCatalog.SessionContext context, Namespace namespace)	Checks whether the Namespace exists.
protected org.apache.iceberg.rest.RESTTableOperations	newTableOps(RESTClient restClient, String path, Supplier<Map<String,String>> readHeaders, Supplier<Map<String,String>> mutationHeaderSupplier, FileIO fileIO, org.apache.iceberg.rest.RESTTableOperations.UpdateType updateType, List<MetadataUpdate> createChanges, TableMetadata current, Set<Endpoint> supportedEndpoints)	Create a new RESTTableOperations instance for transaction-based operations (create or replace).
protected org.apache.iceberg.rest.RESTTableOperations	newTableOps(RESTClient restClient, String path, Supplier<Map<String,String>> readHeaders, Supplier<Map<String,String>> mutationHeaderSupplier, FileIO fileIO, TableMetadata current, Set<Endpoint> supportedEndpoints)	Create a new RESTTableOperations instance for simple table operations.
protected org.apache.iceberg.rest.RESTViewOperations	newViewOps(RESTClient restClient, String path, Supplier<Map<String,String>> readHeaders, Supplier<Map<String,String>> mutationHeaderSupplier, ViewMetadata current, Set<Endpoint> supportedEndpoints)	Create a new RESTViewOperations instance.
boolean	purgeTable(SessionCatalog.SessionContext context, TableIdentifier identifier)	Drop a table and request that files are immediately deleted.
Table	registerTable(SessionCatalog.SessionContext context, TableIdentifier ident, String metadataFileLocation)	Register a table if it does not exist.
Table	registerTable(SessionCatalog.SessionContext context, TableIdentifier ident, String metadataFileLocation, boolean overwrite)	Register a table.
View	registerView(SessionCatalog.SessionContext context, TableIdentifier ident, String metadataFileLocation)	Register a view if it does not exist.
void	renameTable(SessionCatalog.SessionContext context, TableIdentifier from, TableIdentifier to)	Rename a table.
void	renameView(SessionCatalog.SessionContext context, TableIdentifier from, TableIdentifier to)	Rename a view.
void	setConf(Object newConf)
boolean	tableExists(SessionCatalog.SessionContext context, TableIdentifier identifier)	Check whether table exists.
boolean	updateNamespaceMetadata(SessionCatalog.SessionContext context, Namespace ns, Map<String,String> updates, Set<String> removals)	Set a collection of properties on a namespace in the catalog.
boolean	viewExists(SessionCatalog.SessionContext context, TableIdentifier identifier)	Check whether view exists.

Methods inherited from class org.apache.iceberg.catalog.BaseViewSessionCatalog

asViewCatalog

Methods inherited from class org.apache.iceberg.catalog.BaseSessionCatalog

asCatalog, name, properties, withContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.catalog.SessionCatalog

createNamespace, listNamespaces

Methods inherited from interface org.apache.iceberg.catalog.ViewSessionCatalog

invalidateView, name

Field Details

REST_PAGE_SIZE

@Deprecated
public static final String REST_PAGE_SIZE

Deprecated.

will be removed in 1.12.0. Use RESTCatalogProperties.PAGE_SIZE instead.

See Also:

• Constant Field Values

Constructor Details

RESTSessionCatalog

public RESTSessionCatalog()

RESTSessionCatalog

public RESTSessionCatalog(Function<Map<String,String>,RESTClient> clientBuilder,
 BiFunction<SessionCatalog.SessionContext,Map<String,String>,FileIO> ioBuilder)

Method Details

initialize

public void initialize(String name,
 Map<String,String> unresolved)

Description copied from interface: SessionCatalog

Initialize given a custom name and a map of catalog properties.

Specified by:

initialize in interface SessionCatalog

Specified by:

initialize in interface ViewSessionCatalog

Overrides:

initialize in class BaseSessionCatalog

Parameters:

name - a custom name for the catalog

unresolved - catalog properties

createTableCache

protected org.apache.iceberg.rest.RESTTableCache createTableCache(Map<String,String> props)

setConf

public void setConf(Object newConf)

Specified by:

setConf in interface Configurable<Object>

listTables

public List<TableIdentifier> listTables(SessionCatalog.SessionContext context,
 Namespace ns)

Description copied from interface: SessionCatalog

Return all the identifiers under this namespace.

Specified by:

listTables in interface SessionCatalog

Parameters:

context - session context

ns - a namespace

Returns:

a list of identifiers for tables

dropTable

public boolean dropTable(SessionCatalog.SessionContext context,
 TableIdentifier identifier)

Description copied from interface: SessionCatalog

Drop a table, without requesting that files are immediately deleted.

Data and metadata files should be deleted according to the catalog's policy.

Specified by:

dropTable in interface SessionCatalog

Parameters:

context - session context

identifier - a table identifier

Returns:

true if the table was dropped, false if the table did not exist

purgeTable

public boolean purgeTable(SessionCatalog.SessionContext context,
 TableIdentifier identifier)

Description copied from interface: SessionCatalog

Drop a table and request that files are immediately deleted.

Specified by:

purgeTable in interface SessionCatalog

Parameters:

context - session context

identifier - a table identifier

Returns:

true if the table was dropped and purged, false if the table did not exist

renameTable

public void renameTable(SessionCatalog.SessionContext context,
 TableIdentifier from,
 TableIdentifier to)

Description copied from interface: SessionCatalog

Rename a table.

Specified by:

renameTable in interface SessionCatalog

Parameters:

context - session context

from - identifier of the table to rename

to - new table name

tableExists

public boolean tableExists(SessionCatalog.SessionContext context,
 TableIdentifier identifier)

Description copied from interface: SessionCatalog

Check whether table exists.

Specified by:

tableExists in interface SessionCatalog

Parameters:

context - session context

identifier - a table identifier

Returns:

true if the table exists, false otherwise

loadTable

public Table loadTable(SessionCatalog.SessionContext context,
 TableIdentifier identifier)

Description copied from interface: SessionCatalog

Load a table.

Specified by:

loadTable in interface SessionCatalog

Parameters:

context - session context

identifier - a table identifier

Returns:

instance of Table implementation referred by tableIdentifier

buildTable

public Catalog.TableBuilder buildTable(SessionCatalog.SessionContext context,
 TableIdentifier identifier,
 Schema schema)

Description copied from interface: SessionCatalog

Create a builder to create a table or start a create/replace transaction.

Specified by:

buildTable in interface SessionCatalog

Parameters:

context - session context

identifier - a table identifier

schema - a schema

Returns:

the builder to create a table or start a create/replace transaction

invalidateTable

public void invalidateTable(SessionCatalog.SessionContext context,
 TableIdentifier ident)

Description copied from interface: SessionCatalog

Invalidate cached table metadata from current catalog.

If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing.

Specified by:

invalidateTable in interface SessionCatalog

Parameters:

context - session context

ident - a table identifier

registerTable

public Table registerTable(SessionCatalog.SessionContext context,
 TableIdentifier ident,
 String metadataFileLocation)

Description copied from interface: SessionCatalog

Register a table if it does not exist.

For overwrite support, see SessionCatalog.registerTable(SessionContext, TableIdentifier, String, boolean).

Specified by:

registerTable in interface SessionCatalog

Parameters:

context - session context

ident - a table identifier

metadataFileLocation - the location of a metadata file

Returns:

a Table instance

registerTable

public Table registerTable(SessionCatalog.SessionContext context,
 TableIdentifier ident,
 String metadataFileLocation,
 boolean overwrite)

Description copied from interface: SessionCatalog

Register a table.

Specified by:

registerTable in interface SessionCatalog

Parameters:

context - session context

ident - a table identifier

metadataFileLocation - the location of a metadata file

overwrite - whether to overwrite an existing table registration

Returns:

a Table instance

createNamespace

public void createNamespace(SessionCatalog.SessionContext context,
 Namespace namespace,
 Map<String,String> metadata)

Description copied from interface: SessionCatalog

Create a namespace in the catalog.

Specified by:

createNamespace in interface SessionCatalog

Parameters:

context - session context

namespace - a namespace

metadata - a string Map of properties for the given namespace

listNamespaces

public List<Namespace> listNamespaces(SessionCatalog.SessionContext context,
 Namespace namespace)

Description copied from interface: SessionCatalog

List child namespaces from the namespace.

For two existing tables named 'a.b.c.table' and 'a.b.d.table', this method returns:

• Given: Namespace.empty()
• Returns: Namespace.of("a")

• Given: Namespace.of("a")
• Returns: Namespace.of("a", "b")

• Given: Namespace.of("a", "b")
• Returns: Namespace.of("a", "b", "c") and Namespace.of("a", "b", "d")

• Given: Namespace.of("a", "b", "c")
• Returns: empty list, because there are no child namespaces

Specified by:

listNamespaces in interface SessionCatalog

Parameters:

context - session context

namespace - a namespace

Returns:

a List of child Namespace names from the given namespace

namespaceExists

public boolean namespaceExists(SessionCatalog.SessionContext context,
 Namespace namespace)

Description copied from interface: SessionCatalog

Checks whether the Namespace exists.

Specified by:

namespaceExists in interface SessionCatalog

Parameters:

context - session context

namespace - a namespace

Returns:

true if the Namespace exists, false otherwise

loadNamespaceMetadata

public Map<String,String> loadNamespaceMetadata(SessionCatalog.SessionContext context,
 Namespace ns)

Description copied from interface: SessionCatalog

Load metadata properties for a namespace.

Specified by:

loadNamespaceMetadata in interface SessionCatalog

Parameters:

context - session context

ns - a namespace

Returns:

a string map of properties for the given namespace

dropNamespace

public boolean dropNamespace(SessionCatalog.SessionContext context,
 Namespace ns)

Description copied from interface: SessionCatalog

Drop a namespace. If the namespace exists and was dropped, this will return true.

Specified by:

dropNamespace in interface SessionCatalog

Parameters:

context - session context

ns - a namespace

Returns:

true if the namespace was dropped, false otherwise.

updateNamespaceMetadata

public boolean updateNamespaceMetadata(SessionCatalog.SessionContext context,
 Namespace ns,
 Map<String,String> updates,
 Set<String> removals)

Description copied from interface: SessionCatalog

Set a collection of properties on a namespace in the catalog.

Properties that are not in the given map are not modified or removed by this method.

Specified by:

updateNamespaceMetadata in interface SessionCatalog

Parameters:

context - session context

ns - a namespace

updates - properties to set for the namespace

removals - properties to remove from the namespace

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

newTableOps

protected org.apache.iceberg.rest.RESTTableOperations newTableOps(RESTClient restClient,
 String path,
 Supplier<Map<String,String>> readHeaders,
 Supplier<Map<String,String>> mutationHeaderSupplier,
 FileIO fileIO,
 TableMetadata current,
 Set<Endpoint> supportedEndpoints)

Create a new RESTTableOperations instance for simple table operations.

This method can be overridden in subclasses to provide custom table operations implementations.

Parameters:

restClient - the REST client to use for communicating with the catalog server

path - the REST path for the table

readHeaders - a supplier for additional HTTP headers to include in read requests (GET/HEAD)

mutationHeaderSupplier - a supplier for additional HTTP headers to include in mutation requests (POST/DELETE)

fileIO - the FileIO implementation for reading and writing table metadata and data files

current - the current table metadata

supportedEndpoints - the set of supported REST endpoints

Returns:

a new RESTTableOperations instance

newTableOps

protected org.apache.iceberg.rest.RESTTableOperations newTableOps(RESTClient restClient,
 String path,
 Supplier<Map<String,String>> readHeaders,
 Supplier<Map<String,String>> mutationHeaderSupplier,
 FileIO fileIO,
 org.apache.iceberg.rest.RESTTableOperations.UpdateType updateType,
 List<MetadataUpdate> createChanges,
 TableMetadata current,
 Set<Endpoint> supportedEndpoints)

Create a new RESTTableOperations instance for transaction-based operations (create or replace).

This method can be overridden in subclasses to provide custom table operations implementations for transaction-based operations.

Parameters:

restClient - the REST client to use for communicating with the catalog server

path - the REST path for the table

readHeaders - a supplier for additional HTTP headers to include in read requests (GET/HEAD)

mutationHeaderSupplier - a supplier for additional HTTP headers to include in mutation requests (POST/DELETE)

fileIO - the FileIO implementation for reading and writing table metadata and data files

updateType - the RESTTableOperations.UpdateType being performed

createChanges - the list of metadata updates to apply during table creation or replacement

current - the current table metadata (may be null for CREATE operations)

supportedEndpoints - the set of supported REST endpoints

Returns:

a new RESTTableOperations instance

newViewOps

protected org.apache.iceberg.rest.RESTViewOperations newViewOps(RESTClient restClient,
 String path,
 Supplier<Map<String,String>> readHeaders,
 Supplier<Map<String,String>> mutationHeaderSupplier,
 ViewMetadata current,
 Set<Endpoint> supportedEndpoints)

Create a new RESTViewOperations instance.

This method can be overridden in subclasses to provide custom view operations implementations.

Parameters:

restClient - the REST client to use for communicating with the catalog server

path - the REST path for the view

readHeaders - a supplier for additional HTTP headers to include in read requests (GET/HEAD)

mutationHeaderSupplier - a supplier for additional HTTP headers to include in mutation requests (POST/DELETE)

current - the current view metadata

supportedEndpoints - the set of supported REST endpoints

Returns:

a new RESTViewOperations instance

commitTransaction

public void commitTransaction(SessionCatalog.SessionContext context,
 List<TableCommit> commits)

listViews

public List<TableIdentifier> listViews(SessionCatalog.SessionContext context,
 Namespace namespace)

Description copied from interface: ViewSessionCatalog

Return all the identifiers under this namespace.

Specified by:

listViews in interface ViewSessionCatalog

Parameters:

namespace - a namespace

Returns:

a list of identifiers for views

viewExists

public boolean viewExists(SessionCatalog.SessionContext context,
 TableIdentifier identifier)

Description copied from interface: ViewSessionCatalog

Check whether view exists.

Specified by:

viewExists in interface ViewSessionCatalog

Parameters:

identifier - a view identifier

Returns:

true if the view exists, false otherwise

loadView

public View loadView(SessionCatalog.SessionContext context,
 TableIdentifier identifier)

Description copied from interface: ViewSessionCatalog

Load a view.

Specified by:

loadView in interface ViewSessionCatalog

Parameters:

identifier - a view identifier

Returns:

instance of View implementation referred by the identifier

buildView

public ViewBuilder buildView(SessionCatalog.SessionContext context,
 TableIdentifier identifier)

Description copied from interface: ViewSessionCatalog

Instantiate a builder to create or replace a SQL view.

Specified by:

buildView in interface ViewSessionCatalog

Parameters:

identifier - a view identifier

Returns:

a view builder

dropView

public boolean dropView(SessionCatalog.SessionContext context,
 TableIdentifier identifier)

Description copied from interface: ViewSessionCatalog

Drop a view.

Specified by:

dropView in interface ViewSessionCatalog

Parameters:

identifier - a view identifier

Returns:

true if the view was dropped, false if the view did not exist

renameView

public void renameView(SessionCatalog.SessionContext context,
 TableIdentifier from,
 TableIdentifier to)

Description copied from interface: ViewSessionCatalog

Rename a view.

Specified by:

renameView in interface ViewSessionCatalog

Parameters:

from - identifier of the view to rename

to - new view identifier

registerView

public View registerView(SessionCatalog.SessionContext context,
 TableIdentifier ident,
 String metadataFileLocation)

Description copied from interface: ViewSessionCatalog

Register a view if it does not exist.

Specified by:

registerView in interface ViewSessionCatalog

Parameters:

context - session context

ident - a view identifier

metadataFileLocation - the location of a metadata file

Returns:

a View instance