Class RESTCatalog
- java.lang.Object
-
- org.apache.iceberg.rest.RESTCatalog
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
,Catalog
,SupportsNamespaces
,Configurable<java.lang.Object>
public class RESTCatalog extends java.lang.Object implements Catalog, SupportsNamespaces, Configurable<java.lang.Object>, java.io.Closeable
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.apache.iceberg.catalog.Catalog
Catalog.TableBuilder
-
-
Constructor Summary
Constructors Constructor Description RESTCatalog()
RESTCatalog(java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,RESTClient> clientBuilder)
RESTCatalog(SessionCatalog.SessionContext context, java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,RESTClient> clientBuilder)
-
Method Summary
All Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description Catalog.TableBuilder
buildTable(TableIdentifier ident, Schema schema)
/** Instantiate a builder to either create a table or start a create/replace transaction.void
close()
void
createNamespace(Namespace ns, java.util.Map<java.lang.String,java.lang.String> props)
Create a namespace in the catalog.Table
createTable(TableIdentifier identifier, Schema schema)
Create an unpartitioned table.Table
createTable(TableIdentifier ident, Schema schema, PartitionSpec spec)
Create a table.Table
createTable(TableIdentifier ident, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> props)
Create a table.Table
createTable(TableIdentifier ident, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> props)
Create a table.boolean
dropNamespace(Namespace ns)
Drop a namespace.boolean
dropTable(TableIdentifier ident)
Drop a table and delete all data and metadata files.boolean
dropTable(TableIdentifier ident, boolean purge)
Drop a table; optionally delete data and metadata files.void
initialize(java.lang.String name, java.util.Map<java.lang.String,java.lang.String> props)
Initialize a catalog given a custom name and a map of catalog properties.void
invalidateTable(TableIdentifier ident)
Invalidate cached table metadata from current catalog.java.util.List<Namespace>
listNamespaces(Namespace ns)
List namespaces from the namespace.java.util.List<TableIdentifier>
listTables(Namespace ns)
Return all the identifiers under this namespace.java.util.Map<java.lang.String,java.lang.String>
loadNamespaceMetadata(Namespace ns)
Load metadata properties for a namespace.Table
loadTable(TableIdentifier ident)
Load a table.java.lang.String
name()
Return the name for this catalog.Transaction
newCreateTableTransaction(TableIdentifier identifier, Schema schema)
Start a transaction to create a table.Transaction
newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec)
Start a transaction to create a table.Transaction
newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> props)
Start a transaction to create a table.Transaction
newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> props)
Start a transaction to create a table.Transaction
newReplaceTableTransaction(TableIdentifier ident, Schema schema, boolean orCreate)
Start a transaction to replace a table.Transaction
newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, boolean orCreate)
Start a transaction to replace a table.Transaction
newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> props, boolean orCreate)
Start a transaction to replace a table.Transaction
newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> props, boolean orCreate)
Start a transaction to replace a table.java.util.Map<java.lang.String,java.lang.String>
properties()
Table
registerTable(TableIdentifier ident, java.lang.String metadataFileLocation)
Register a table with the catalog if it does not exist.boolean
removeProperties(Namespace ns, java.util.Set<java.lang.String> props)
Remove a set of property keys from a namespace in the catalog.void
renameTable(TableIdentifier from, TableIdentifier to)
Rename a table.void
setConf(java.lang.Object conf)
void
setConf(org.apache.hadoop.conf.Configuration conf)
Deprecated.will be removed in 1.3.0; usesetConf(Object)
boolean
setProperties(Namespace ns, java.util.Map<java.lang.String,java.lang.String> props)
Set a collection of properties on a namespace in the catalog.boolean
tableExists(TableIdentifier ident)
Check whether table exists.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.iceberg.catalog.SupportsNamespaces
createNamespace, listNamespaces, namespaceExists
-
-
-
-
Constructor Detail
-
RESTCatalog
public RESTCatalog()
-
RESTCatalog
public RESTCatalog(java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,RESTClient> clientBuilder)
-
RESTCatalog
public RESTCatalog(SessionCatalog.SessionContext context, java.util.function.Function<java.util.Map<java.lang.String,java.lang.String>,RESTClient> clientBuilder)
-
-
Method Detail
-
initialize
public void initialize(java.lang.String name, java.util.Map<java.lang.String,java.lang.String> props)
Description copied from interface:Catalog
Initialize a catalog given a custom name and a map of catalog properties.A custom Catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.
- Specified by:
initialize
in interfaceCatalog
- Parameters:
name
- a custom name for the catalogprops
- catalog properties
-
name
public java.lang.String name()
Description copied from interface:Catalog
Return the name for this catalog.
-
properties
public java.util.Map<java.lang.String,java.lang.String> properties()
-
listTables
public java.util.List<TableIdentifier> listTables(Namespace ns)
Description copied from interface:Catalog
Return all the identifiers under this namespace.- Specified by:
listTables
in interfaceCatalog
- Parameters:
ns
- a namespace- Returns:
- a list of identifiers for tables
-
tableExists
public boolean tableExists(TableIdentifier ident)
Description copied from interface:Catalog
Check whether table exists.- Specified by:
tableExists
in interfaceCatalog
- Parameters:
ident
- a table identifier- Returns:
- true if the table exists, false otherwise
-
loadTable
public Table loadTable(TableIdentifier ident)
Description copied from interface:Catalog
Load a table.
-
invalidateTable
public void invalidateTable(TableIdentifier ident)
Description copied from interface:Catalog
Invalidate cached table metadata from current catalog.If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing.
- Specified by:
invalidateTable
in interfaceCatalog
- Parameters:
ident
- a table identifier
-
buildTable
public Catalog.TableBuilder buildTable(TableIdentifier ident, Schema schema)
Description copied from interface:Catalog
/** Instantiate a builder to either create a table or start a create/replace transaction.- Specified by:
buildTable
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schema- Returns:
- the builder to create a table or start a create/replace transaction
-
createTable
public Table createTable(TableIdentifier ident, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> props)
Description copied from interface:Catalog
Create a table.- Specified by:
createTable
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition speclocation
- a location for the table; leave null if unspecifiedprops
- a string map of table properties- Returns:
- a Table instance
-
createTable
public Table createTable(TableIdentifier ident, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> props)
Description copied from interface:Catalog
Create a table.- Specified by:
createTable
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition specprops
- a string map of table properties- Returns:
- a Table instance
-
createTable
public Table createTable(TableIdentifier ident, Schema schema, PartitionSpec spec)
Description copied from interface:Catalog
Create a table.- Specified by:
createTable
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition spec- Returns:
- a Table instance
-
createTable
public Table createTable(TableIdentifier identifier, Schema schema)
Description copied from interface:Catalog
Create an unpartitioned table.- Specified by:
createTable
in interfaceCatalog
- Parameters:
identifier
- a table identifierschema
- a schema- Returns:
- a Table instance
-
newCreateTableTransaction
public Transaction newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> props)
Description copied from interface:Catalog
Start a transaction to create a table.- Specified by:
newCreateTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition speclocation
- a location for the table; leave null if unspecifiedprops
- a string map of table properties- Returns:
- a
Transaction
to create the table
-
newCreateTableTransaction
public Transaction newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> props)
Description copied from interface:Catalog
Start a transaction to create a table.- Specified by:
newCreateTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition specprops
- a string map of table properties- Returns:
- a
Transaction
to create the table
-
newCreateTableTransaction
public Transaction newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec)
Description copied from interface:Catalog
Start a transaction to create a table.- Specified by:
newCreateTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition spec- Returns:
- a
Transaction
to create the table
-
newCreateTableTransaction
public Transaction newCreateTableTransaction(TableIdentifier identifier, Schema schema)
Description copied from interface:Catalog
Start a transaction to create a table.- Specified by:
newCreateTableTransaction
in interfaceCatalog
- Parameters:
identifier
- a table identifierschema
- a schema- Returns:
- a
Transaction
to create the table
-
newReplaceTableTransaction
public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.lang.String location, java.util.Map<java.lang.String,java.lang.String> props, boolean orCreate)
Description copied from interface:Catalog
Start a transaction to replace a table.- Specified by:
newReplaceTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition speclocation
- a location for the table; leave null if unspecifiedprops
- a string map of table propertiesorCreate
- whether to create the table if not exists- Returns:
- a
Transaction
to replace the table
-
newReplaceTableTransaction
public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, java.util.Map<java.lang.String,java.lang.String> props, boolean orCreate)
Description copied from interface:Catalog
Start a transaction to replace a table.- Specified by:
newReplaceTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition specprops
- a string map of table propertiesorCreate
- whether to create the table if not exists- Returns:
- a
Transaction
to replace the table
-
newReplaceTableTransaction
public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, boolean orCreate)
Description copied from interface:Catalog
Start a transaction to replace a table.- Specified by:
newReplaceTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaspec
- a partition specorCreate
- whether to create the table if not exists- Returns:
- a
Transaction
to replace the table
-
newReplaceTableTransaction
public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, boolean orCreate)
Description copied from interface:Catalog
Start a transaction to replace a table.- Specified by:
newReplaceTableTransaction
in interfaceCatalog
- Parameters:
ident
- a table identifierschema
- a schemaorCreate
- whether to create the table if not exists- Returns:
- a
Transaction
to replace the table
-
dropTable
public boolean dropTable(TableIdentifier ident)
Description copied from interface:Catalog
Drop a table and delete all data and metadata files.
-
dropTable
public boolean dropTable(TableIdentifier ident, boolean purge)
Description copied from interface:Catalog
Drop a table; optionally delete data and metadata files.If purge is set to true the implementation should delete all data and metadata files.
-
renameTable
public void renameTable(TableIdentifier from, TableIdentifier to)
Description copied from interface:Catalog
Rename a table.- Specified by:
renameTable
in interfaceCatalog
- Parameters:
from
- identifier of the table to renameto
- new table name
-
registerTable
public Table registerTable(TableIdentifier ident, java.lang.String metadataFileLocation)
Description copied from interface:Catalog
Register a table with the catalog if it does not exist.- Specified by:
registerTable
in interfaceCatalog
- Parameters:
ident
- a table identifiermetadataFileLocation
- the location of a metadata file- Returns:
- a Table instance
-
createNamespace
public void createNamespace(Namespace ns, java.util.Map<java.lang.String,java.lang.String> props)
Description copied from interface:SupportsNamespaces
Create a namespace in the catalog.- Specified by:
createNamespace
in interfaceSupportsNamespaces
- Parameters:
ns
- a multi-part namespaceprops
- a string Map of properties for the given namespace
-
listNamespaces
public java.util.List<Namespace> listNamespaces(Namespace ns) throws NoSuchNamespaceException
Description copied from interface:SupportsNamespaces
List namespaces from the namespace.For example, if table a.b.t exists, use 'SELECT NAMESPACE IN a' this method must return Namepace.of("a","b")
Namespace
.- Specified by:
listNamespaces
in interfaceSupportsNamespaces
- Returns:
- a List of namespace
Namespace
names - Throws:
NoSuchNamespaceException
- If the namespace does not exist (optional)
-
loadNamespaceMetadata
public java.util.Map<java.lang.String,java.lang.String> loadNamespaceMetadata(Namespace ns) throws NoSuchNamespaceException
Description copied from interface:SupportsNamespaces
Load metadata properties for a namespace.- Specified by:
loadNamespaceMetadata
in interfaceSupportsNamespaces
- Parameters:
ns
- a namespace.Namespace
- Returns:
- a string map of properties for the given namespace
- Throws:
NoSuchNamespaceException
- If the namespace does not exist (optional)
-
dropNamespace
public boolean dropNamespace(Namespace ns) throws NamespaceNotEmptyException
Description copied from interface:SupportsNamespaces
Drop a namespace. If the namespace exists and was dropped, this will return true.- Specified by:
dropNamespace
in interfaceSupportsNamespaces
- Parameters:
ns
- a namespace.Namespace
- Returns:
- true if the namespace was dropped, false otherwise.
- Throws:
NamespaceNotEmptyException
- If the namespace is not empty
-
setProperties
public boolean setProperties(Namespace ns, java.util.Map<java.lang.String,java.lang.String> props) throws NoSuchNamespaceException
Description copied from interface:SupportsNamespaces
Set a collection of properties on a namespace in the catalog.Properties that are not in the given map are not modified or removed by this method.
- Specified by:
setProperties
in interfaceSupportsNamespaces
- Parameters:
ns
- a namespace.Namespace
props
- a collection of metadata to apply to the namespace- Throws:
NoSuchNamespaceException
- If the namespace does not exist (optional)
-
removeProperties
public boolean removeProperties(Namespace ns, java.util.Set<java.lang.String> props) throws NoSuchNamespaceException
Description copied from interface:SupportsNamespaces
Remove a set of property keys from a namespace in the catalog.Properties that are not in the given set are not modified or removed by this method.
- Specified by:
removeProperties
in interfaceSupportsNamespaces
- Parameters:
ns
- a namespace.Namespace
props
- a collection of metadata to apply to the namespace- Throws:
NoSuchNamespaceException
- If the namespace does not exist (optional)
-
setConf
public void setConf(java.lang.Object conf)
- Specified by:
setConf
in interfaceConfigurable<java.lang.Object>
-
setConf
@Deprecated public void setConf(org.apache.hadoop.conf.Configuration conf)
Deprecated.will be removed in 1.3.0; usesetConf(Object)
-
close
public void close() throws java.io.IOException
- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfacejava.io.Closeable
- Throws:
java.io.IOException
-
-