Class RESTCatalog
- All Implemented Interfaces:
Closeable
,AutoCloseable
,Catalog
,SupportsNamespaces
,ViewCatalog
,Configurable<Object>
-
Nested Class Summary
Nested classes/interfaces inherited from interface org.apache.iceberg.catalog.Catalog
Catalog.TableBuilder
-
Constructor Summary
ConstructorDescriptionRESTCatalog
(Function<Map<String, String>, RESTClient> clientBuilder) RESTCatalog
(SessionCatalog.SessionContext context, Function<Map<String, String>, RESTClient> clientBuilder) -
Method Summary
Modifier and TypeMethodDescriptionbuildTable
(TableIdentifier ident, Schema schema) /** Instantiate a builder to either create a table or start a create/replace transaction.buildView
(TableIdentifier identifier) Instantiate a builder to create or replace a SQL view.void
close()
void
commitTransaction
(List<TableCommit> commits) void
commitTransaction
(TableCommit... commits) void
createNamespace
(Namespace ns, Map<String, String> props) Create a namespace in the catalog.createTable
(TableIdentifier identifier, Schema schema) Create an unpartitioned table.createTable
(TableIdentifier ident, Schema schema, PartitionSpec spec) Create a table.createTable
(TableIdentifier ident, Schema schema, PartitionSpec spec, String location, Map<String, String> props) Create a table.createTable
(TableIdentifier ident, Schema schema, PartitionSpec spec, Map<String, String> props) Create a table.boolean
Drop a namespace.boolean
dropTable
(TableIdentifier ident) Drop a table and delete all data and metadata files.boolean
dropTable
(TableIdentifier ident, boolean purge) Drop a table; optionally delete data and metadata files.boolean
dropView
(TableIdentifier identifier) Drop a view.void
initialize
(String name, Map<String, String> props) Initialize a catalog given a custom name and a map of catalog properties.void
invalidateTable
(TableIdentifier ident) Invalidate cached table metadata from current catalog.void
invalidateView
(TableIdentifier identifier) Invalidate cached view metadata from current catalog.List child namespaces from the namespace.listTables
(Namespace ns) Return all the identifiers under this namespace.Return all the identifiers under this namespace.Load metadata properties for a namespace.loadTable
(TableIdentifier ident) Load a table.loadView
(TableIdentifier identifier) Load a view.name()
Return the name for this catalog.newCreateTableTransaction
(TableIdentifier identifier, Schema schema) Start a transaction to create a table.newCreateTableTransaction
(TableIdentifier ident, Schema schema, PartitionSpec spec) Start a transaction to create a table.newCreateTableTransaction
(TableIdentifier ident, Schema schema, PartitionSpec spec, String location, Map<String, String> props) Start a transaction to create a table.newCreateTableTransaction
(TableIdentifier ident, Schema schema, PartitionSpec spec, Map<String, String> props) Start a transaction to create a table.newReplaceTableTransaction
(TableIdentifier ident, Schema schema, boolean orCreate) Start a transaction to replace a table.newReplaceTableTransaction
(TableIdentifier ident, Schema schema, PartitionSpec spec, boolean orCreate) Start a transaction to replace a table.newReplaceTableTransaction
(TableIdentifier ident, Schema schema, PartitionSpec spec, String location, Map<String, String> props, boolean orCreate) Start a transaction to replace a table.newReplaceTableTransaction
(TableIdentifier ident, Schema schema, PartitionSpec spec, Map<String, String> props, boolean orCreate) Start a transaction to replace a table.registerTable
(TableIdentifier ident, String metadataFileLocation) Register a table with the catalog if it does not exist.boolean
removeProperties
(Namespace ns, Set<String> props) Remove a set of property keys from a namespace in the catalog.void
renameTable
(TableIdentifier from, TableIdentifier to) Rename a table.void
renameView
(TableIdentifier from, TableIdentifier to) Rename a view.void
boolean
setProperties
(Namespace ns, Map<String, String> props) Set a collection of properties on a namespace in the catalog.boolean
tableExists
(TableIdentifier ident) Check whether table exists.boolean
viewExists
(TableIdentifier identifier) Check whether view exists.Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.iceberg.catalog.SupportsNamespaces
createNamespace, listNamespaces, namespaceExists
-
Constructor Details
-
RESTCatalog
public RESTCatalog() -
RESTCatalog
-
RESTCatalog
public RESTCatalog(SessionCatalog.SessionContext context, Function<Map<String, String>, RESTClient> clientBuilder)
-
-
Method Details
-
initialize
Description copied from interface:Catalog
Initialize a catalog given a custom name and a map of catalog properties.A custom Catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.
- Specified by:
initialize
in interfaceCatalog
- Specified by:
initialize
in interfaceViewCatalog
- Parameters:
name
- a custom name for the catalogprops
- catalog properties
-
name
Description copied from interface:Catalog
Return the name for this catalog.- Specified by:
name
in interfaceCatalog
- Specified by:
name
in interfaceViewCatalog
- Returns:
- this catalog's name
-
properties
-
listTables
Description copied from interface:Catalog
Return all the identifiers under this namespace.- Specified by:
listTables
in interfaceCatalog
- Parameters:
ns
- a namespace- Returns:
- a list of identifiers for tables
-
tableExists
Description copied from interface:Catalog
Check whether table exists.- Specified by:
tableExists
in interfaceCatalog
- Parameters:
ident
- a table identifier- Returns:
- true if the table exists, false otherwise
-
loadTable
Description copied from interface:Catalog
Load a table. -
invalidateTable
Description copied from interface:Catalog
Invalidate cached table metadata from current catalog.If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing.
- Specified by:
invalidateTable
in interfaceCatalog
- Parameters:
ident
- a table identifier
-
buildTable
Description copied from interface:Catalog
/** Instantiate a builder to either create a table or start a create/replace transaction.- Specified by:
buildTable
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schema- Returns:
- the builder to create a table or start a create/replace transaction
-
createTable
public Table createTable(TableIdentifier ident, Schema schema, PartitionSpec spec, String location, Map<String, String> props) Description copied from interface:Catalog
Create a table.- Specified by:
createTable
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition speclocation
- a location for the table; leave null if unspecifiedprops
- a string map of table properties- Returns:
- a Table instance
-
createTable
public Table createTable(TableIdentifier ident, Schema schema, PartitionSpec spec, Map<String, String> props) Description copied from interface:Catalog
Create a table.- Specified by:
createTable
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition specprops
- a string map of table properties- Returns:
- a Table instance
-
createTable
Description copied from interface:Catalog
Create a table.- Specified by:
createTable
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition spec- Returns:
- a Table instance
-
createTable
Description copied from interface:Catalog
Create an unpartitioned table.- Specified by:
createTable
in interfaceCatalog
- Parameters:
identifier
- a table identifierschema
- a schema- Returns:
- a Table instance
-
newCreateTableTransaction
public Transaction newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, String location, Map<String, String> props) Description copied from interface:Catalog
Start a transaction to create a table.- Specified by:
newCreateTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition speclocation
- a location for the table; leave null if unspecifiedprops
- a string map of table properties- Returns:
- a
Transaction
to create the table
-
newCreateTableTransaction
public Transaction newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, Map<String, String> props) Description copied from interface:Catalog
Start a transaction to create a table.- Specified by:
newCreateTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition specprops
- a string map of table properties- Returns:
- a
Transaction
to create the table
-
newCreateTableTransaction
public Transaction newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec) Description copied from interface:Catalog
Start a transaction to create a table.- Specified by:
newCreateTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition spec- Returns:
- a
Transaction
to create the table
-
newCreateTableTransaction
Description copied from interface:Catalog
Start a transaction to create a table.- Specified by:
newCreateTableTransaction
in interfaceCatalog
- Parameters:
identifier
- a table identifierschema
- a schema- Returns:
- a
Transaction
to create the table
-
newReplaceTableTransaction
public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, String location, Map<String, String> props, boolean orCreate) Description copied from interface:Catalog
Start a transaction to replace a table.- Specified by:
newReplaceTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition speclocation
- a location for the table; leave null if unspecifiedprops
- a string map of table propertiesorCreate
- whether to create the table if not exists- Returns:
- a
Transaction
to replace the table
-
newReplaceTableTransaction
public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, Map<String, String> props, boolean orCreate) Description copied from interface:Catalog
Start a transaction to replace a table.- Specified by:
newReplaceTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition specprops
- a string map of table propertiesorCreate
- whether to create the table if not exists- Returns:
- a
Transaction
to replace the table
-
newReplaceTableTransaction
public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, boolean orCreate) Description copied from interface:Catalog
Start a transaction to replace a table.- Specified by:
newReplaceTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition specorCreate
- whether to create the table if not exists- Returns:
- a
Transaction
to replace the table
-
newReplaceTableTransaction
public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, boolean orCreate) Description copied from interface:Catalog
Start a transaction to replace a table.- Specified by:
newReplaceTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaorCreate
- whether to create the table if not exists- Returns:
- a
Transaction
to replace the table
-
dropTable
Description copied from interface:Catalog
Drop a table and delete all data and metadata files. -
dropTable
Description copied from interface:Catalog
Drop a table; optionally delete data and metadata files.If purge is set to true the implementation should delete all data and metadata files.
-
renameTable
Description copied from interface:Catalog
Rename a table.- Specified by:
renameTable
in interfaceCatalog
- Parameters:
from
- identifier of the table to renameto
- new table name
-
registerTable
Description copied from interface:Catalog
Register a table with the catalog if it does not exist.- Specified by:
registerTable
in interfaceCatalog
- Parameters:
ident
- a table identifiermetadataFileLocation
- the location of a metadata file- Returns:
- a Table instance
-
createNamespace
Description copied from interface:SupportsNamespaces
Create a namespace in the catalog.- Specified by:
createNamespace
in interfaceSupportsNamespaces
- Parameters:
ns
- a multi-part namespaceprops
- a string Map of properties for the given namespace
-
listNamespaces
Description copied from interface:SupportsNamespaces
List child namespaces from the namespace.For two existing tables named 'a.b.c.table' and 'a.b.d.table', this method returns:
- Given:
Namespace.empty()
- Returns:
Namespace.of("a")
- Given:
Namespace.of("a")
- Returns:
Namespace.of("a", "b")
- Given:
Namespace.of("a", "b")
- Returns:
Namespace.of("a", "b", "c")
andNamespace.of("a", "b", "d")
- Given:
Namespace.of("a", "b", "c")
- Returns: empty list, because there are no child namespaces
- Specified by:
listNamespaces
in interfaceSupportsNamespaces
- Returns:
- a List of child
Namespace
names from the given namespace - Throws:
NoSuchNamespaceException
- If the namespace does not exist (optional)
- Given:
-
loadNamespaceMetadata
Description copied from interface:SupportsNamespaces
Load metadata properties for a namespace.- Specified by:
loadNamespaceMetadata
in interfaceSupportsNamespaces
- Parameters:
ns
- a namespace.Namespace
- Returns:
- a string map of properties for the given namespace
- Throws:
NoSuchNamespaceException
- If the namespace does not exist (optional)
-
dropNamespace
Description copied from interface:SupportsNamespaces
Drop a namespace. If the namespace exists and was dropped, this will return true.- Specified by:
dropNamespace
in interfaceSupportsNamespaces
- Parameters:
ns
- a namespace.Namespace
- Returns:
- true if the namespace was dropped, false otherwise.
- Throws:
NamespaceNotEmptyException
- If the namespace is not empty
-
setProperties
public boolean setProperties(Namespace ns, Map<String, String> props) throws NoSuchNamespaceExceptionDescription copied from interface:SupportsNamespaces
Set a collection of properties on a namespace in the catalog.Properties that are not in the given map are not modified or removed by this method.
- Specified by:
setProperties
in interfaceSupportsNamespaces
- Parameters:
ns
- a namespace.Namespace
props
- a collection of metadata to apply to the namespace- Throws:
NoSuchNamespaceException
- If the namespace does not exist (optional)
-
removeProperties
Description copied from interface:SupportsNamespaces
Remove a set of property keys from a namespace in the catalog.Properties that are not in the given set are not modified or removed by this method.
- Specified by:
removeProperties
in interfaceSupportsNamespaces
- Parameters:
ns
- a namespace.Namespace
props
- a collection of metadata to apply to the namespace- Throws:
NoSuchNamespaceException
- If the namespace does not exist (optional)
-
setConf
- Specified by:
setConf
in interfaceConfigurable<Object>
-
close
- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Throws:
IOException
-
commitTransaction
-
commitTransaction
-
listViews
Description copied from interface:ViewCatalog
Return all the identifiers under this namespace.- Specified by:
listViews
in interfaceViewCatalog
- Parameters:
namespace
- a namespace- Returns:
- a list of identifiers for views
-
loadView
Description copied from interface:ViewCatalog
Load a view.- Specified by:
loadView
in interfaceViewCatalog
- Parameters:
identifier
- a view identifier- Returns:
- instance of
View
implementation referred by the identifier
-
buildView
Description copied from interface:ViewCatalog
Instantiate a builder to create or replace a SQL view.- Specified by:
buildView
in interfaceViewCatalog
- Parameters:
identifier
- a view identifier- Returns:
- a view builder
-
dropView
Description copied from interface:ViewCatalog
Drop a view.- Specified by:
dropView
in interfaceViewCatalog
- Parameters:
identifier
- a view identifier- Returns:
- true if the view was dropped, false if the view did not exist
-
renameView
Description copied from interface:ViewCatalog
Rename a view.- Specified by:
renameView
in interfaceViewCatalog
- Parameters:
from
- identifier of the view to renameto
- new view identifier
-
viewExists
Description copied from interface:ViewCatalog
Check whether view exists.- Specified by:
viewExists
in interfaceViewCatalog
- Parameters:
identifier
- a view identifier- Returns:
- true if the view exists, false otherwise
-
invalidateView
Description copied from interface:ViewCatalog
Invalidate cached view metadata from current catalog.If the view is already loaded or cached, drop cached data. If the view does not exist or is not cached, do nothing.
- Specified by:
invalidateView
in interfaceViewCatalog
- Parameters:
identifier
- a view identifier
-