Package org.apache.iceberg.rest

Class RESTCatalog

java.lang.Object
org.apache.iceberg.rest.RESTCatalog
All Implemented Interfaces:
Closeable, AutoCloseable, Catalog, SupportsNamespaces, ViewCatalog, Configurable<Object>
public class RESTCatalog extends Object implements Catalog, ViewCatalog, SupportsNamespaces, Configurable<Object>, Closeable

  • Constructor Details

  • Method Details

    • initialize

      public void initialize(String name, Map<String,String> props)
      Description copied from interface: Catalog
      Initialize a catalog given a custom name and a map of catalog properties.

      A custom Catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.

      Specified by:
      initialize in interface Catalog
      Specified by:
      initialize in interface ViewCatalog
      Parameters:
      name - a custom name for the catalog
      props - catalog properties

    • name

      public String name()
      Description copied from interface: Catalog
      Return the name for this catalog.
      Specified by:
      name in interface Catalog
      Specified by:
      name in interface ViewCatalog
      Returns:
      this catalog's name

    • properties

      public Map<String,String> properties()

    • listTables

      public List<TableIdentifier> listTables(Namespace ns)
      Description copied from interface: Catalog
      Return all the identifiers under this namespace.
      Specified by:
      listTables in interface Catalog
      Parameters:
      ns - a namespace
      Returns:
      a list of identifiers for tables

    • tableExists

      public boolean tableExists(TableIdentifier ident)
      Description copied from interface: Catalog
      Check whether table exists.
      Specified by:
      tableExists in interface Catalog
      Parameters:
      ident - a table identifier
      Returns:
      true if the table exists, false otherwise

    • loadTable

      public Table loadTable(TableIdentifier ident)
      Description copied from interface: Catalog
      Load a table.
      Specified by:
      loadTable in interface Catalog
      Parameters:
      ident - a table identifier
      Returns:
      instance of Table implementation referred by tableIdentifier

    • invalidateTable

      public void invalidateTable(TableIdentifier ident)
      Description copied from interface: Catalog
      Invalidate cached table metadata from current catalog.

      If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing.

      Specified by:
      invalidateTable in interface Catalog
      Parameters:
      ident - a table identifier

    • buildTable

      public Catalog.TableBuilder buildTable(TableIdentifier ident, Schema schema)
      Description copied from interface: Catalog
      Instantiate a builder to either create a table or start a create/replace transaction.
      Specified by:
      buildTable in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      Returns:
      the builder to create a table or start a create/replace transaction

    • createTable

      public Table createTable(TableIdentifier ident, Schema schema, PartitionSpec spec, String location, Map<String,String> props)
      Description copied from interface: Catalog
      Create a table.
      Specified by:
      createTable in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      spec - a partition spec
      location - a location for the table; leave null if unspecified
      props - a string map of table properties
      Returns:
      a Table instance

    • createTable

      public Table createTable(TableIdentifier ident, Schema schema, PartitionSpec spec, Map<String,String> props)
      Description copied from interface: Catalog
      Create a table.
      Specified by:
      createTable in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      spec - a partition spec
      props - a string map of table properties
      Returns:
      a Table instance

    • createTable

      public Table createTable(TableIdentifier ident, Schema schema, PartitionSpec spec)
      Description copied from interface: Catalog
      Create a table.
      Specified by:
      createTable in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      spec - a partition spec
      Returns:
      a Table instance

    • createTable

      public Table createTable(TableIdentifier identifier, Schema schema)
      Description copied from interface: Catalog
      Create an unpartitioned table.
      Specified by:
      createTable in interface Catalog
      Parameters:
      identifier - a table identifier
      schema - a schema
      Returns:
      a Table instance

    • newCreateTableTransaction

      public Transaction newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, String location, Map<String,String> props)
      Description copied from interface: Catalog
      Start a transaction to create a table.
      Specified by:
      newCreateTableTransaction in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      spec - a partition spec
      location - a location for the table; leave null if unspecified
      props - a string map of table properties
      Returns:
      a Transaction to create the table

    • newCreateTableTransaction

      public Transaction newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, Map<String,String> props)
      Description copied from interface: Catalog
      Start a transaction to create a table.
      Specified by:
      newCreateTableTransaction in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      spec - a partition spec
      props - a string map of table properties
      Returns:
      a Transaction to create the table

    • newCreateTableTransaction

      public Transaction newCreateTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec)
      Description copied from interface: Catalog
      Start a transaction to create a table.
      Specified by:
      newCreateTableTransaction in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      spec - a partition spec
      Returns:
      a Transaction to create the table

    • newCreateTableTransaction

      public Transaction newCreateTableTransaction(TableIdentifier identifier, Schema schema)
      Description copied from interface: Catalog
      Start a transaction to create a table.
      Specified by:
      newCreateTableTransaction in interface Catalog
      Parameters:
      identifier - a table identifier
      schema - a schema
      Returns:
      a Transaction to create the table

    • newReplaceTableTransaction

      public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, String location, Map<String,String> props, boolean orCreate)
      Description copied from interface: Catalog
      Start a transaction to replace a table.
      Specified by:
      newReplaceTableTransaction in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      spec - a partition spec
      location - a location for the table; leave null if unspecified
      props - a string map of table properties
      orCreate - whether to create the table if not exists
      Returns:
      a Transaction to replace the table

    • newReplaceTableTransaction

      public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, Map<String,String> props, boolean orCreate)
      Description copied from interface: Catalog
      Start a transaction to replace a table.
      Specified by:
      newReplaceTableTransaction in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      spec - a partition spec
      props - a string map of table properties
      orCreate - whether to create the table if not exists
      Returns:
      a Transaction to replace the table

    • newReplaceTableTransaction

      public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, PartitionSpec spec, boolean orCreate)
      Description copied from interface: Catalog
      Start a transaction to replace a table.
      Specified by:
      newReplaceTableTransaction in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      spec - a partition spec
      orCreate - whether to create the table if not exists
      Returns:
      a Transaction to replace the table

    • newReplaceTableTransaction

      public Transaction newReplaceTableTransaction(TableIdentifier ident, Schema schema, boolean orCreate)
      Description copied from interface: Catalog
      Start a transaction to replace a table.
      Specified by:
      newReplaceTableTransaction in interface Catalog
      Parameters:
      ident - a table identifier
      schema - a schema
      orCreate - whether to create the table if not exists
      Returns:
      a Transaction to replace the table

    • dropTable

      public boolean dropTable(TableIdentifier ident)
      Description copied from interface: Catalog
      Drop a table and delete all data and metadata files.
      Specified by:
      dropTable in interface Catalog
      Parameters:
      ident - a table identifier
      Returns:
      true if the table was dropped, false if the table did not exist

    • dropTable

      public boolean dropTable(TableIdentifier ident, boolean purge)
      Description copied from interface: Catalog
      Drop a table; optionally delete data and metadata files.

      If purge is set to true the implementation should delete all data and metadata files.

      Specified by:
      dropTable in interface Catalog
      Parameters:
      ident - a table identifier
      purge - if true, delete all data and metadata files in the table
      Returns:
      true if the table was dropped, false if the table did not exist

    • renameTable

      public void renameTable(TableIdentifier from, TableIdentifier to)
      Description copied from interface: Catalog
      Rename a table.
      Specified by:
      renameTable in interface Catalog
      Parameters:
      from - identifier of the table to rename
      to - new table name

    • registerTable

      public Table registerTable(TableIdentifier ident, String metadataFileLocation)
      Description copied from interface: Catalog
      Register a table with the catalog if it does not exist.
      Specified by:
      registerTable in interface Catalog
      Parameters:
      ident - a table identifier
      metadataFileLocation - the location of a metadata file
      Returns:
      a Table instance

    • createNamespace

      public void createNamespace(Namespace ns, Map<String,String> props)
      Description copied from interface: SupportsNamespaces
      Create a namespace in the catalog.
      Specified by:
      createNamespace in interface SupportsNamespaces
      Parameters:
      ns - a multi-part namespace
      props - a string Map of properties for the given namespace

    • listNamespaces

      public List<Namespace> listNamespaces(Namespace ns) throws NoSuchNamespaceException
      Description copied from interface: SupportsNamespaces
      List child namespaces from the namespace.

      For two existing tables named 'a.b.c.table' and 'a.b.d.table', this method returns:

      • Given: Namespace.empty()
      • Returns: Namespace.of("a")
      • Given: Namespace.of("a")
      • Returns: Namespace.of("a", "b")
      • Given: Namespace.of("a", "b")
      • Returns: Namespace.of("a", "b", "c") and Namespace.of("a", "b", "d")
      • Given: Namespace.of("a", "b", "c")
      • Returns: empty list, because there are no child namespaces
      Specified by:
      listNamespaces in interface SupportsNamespaces
      Returns:
      a List of child Namespace names from the given namespace
      Throws:
      NoSuchNamespaceException - If the namespace does not exist (optional)

    • namespaceExists

      public boolean namespaceExists(Namespace namespace)
      Description copied from interface: SupportsNamespaces
      Checks whether the Namespace exists.
      Specified by:
      namespaceExists in interface SupportsNamespaces
      Parameters:
      namespace - a namespace. Namespace
      Returns:
      true if the Namespace exists, false otherwise

    • loadNamespaceMetadata

      public Map<String,String> loadNamespaceMetadata(Namespace ns) throws NoSuchNamespaceException
      Description copied from interface: SupportsNamespaces
      Load metadata properties for a namespace.
      Specified by:
      loadNamespaceMetadata in interface SupportsNamespaces
      Parameters:
      ns - a namespace. Namespace
      Returns:
      a string map of properties for the given namespace
      Throws:
      NoSuchNamespaceException - If the namespace does not exist (optional)

    • dropNamespace

      public boolean dropNamespace(Namespace ns) throws NamespaceNotEmptyException
      Description copied from interface: SupportsNamespaces
      Drop a namespace. If the namespace exists and was dropped, this will return true.
      Specified by:
      dropNamespace in interface SupportsNamespaces
      Parameters:
      ns - a namespace. Namespace
      Returns:
      true if the namespace was dropped, false otherwise.
      Throws:
      NamespaceNotEmptyException - If the namespace is not empty

    • setProperties

      public boolean setProperties(Namespace ns, Map<String,String> props) throws NoSuchNamespaceException
      Description copied from interface: SupportsNamespaces
      Set a collection of properties on a namespace in the catalog.

      Properties that are not in the given map are not modified or removed by this method.

      Specified by:
      setProperties in interface SupportsNamespaces
      Parameters:
      ns - a namespace. Namespace
      props - a collection of metadata to apply to the namespace
      Throws:
      NoSuchNamespaceException - If the namespace does not exist (optional)

    • removeProperties

      public boolean removeProperties(Namespace ns, Set<String> props) throws NoSuchNamespaceException
      Description copied from interface: SupportsNamespaces
      Remove a set of property keys from a namespace in the catalog.

      Properties that are not in the given set are not modified or removed by this method.

      Specified by:
      removeProperties in interface SupportsNamespaces
      Parameters:
      ns - a namespace. Namespace
      props - a collection of metadata to apply to the namespace
      Throws:
      NoSuchNamespaceException - If the namespace does not exist (optional)

    • setConf

      public void setConf(Object conf)
      Specified by:
      setConf in interface Configurable<Object>

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException

    • commitTransaction

      public void commitTransaction(List<TableCommit> commits)

    • commitTransaction

      public void commitTransaction(TableCommit... commits)

    • listViews

      public List<TableIdentifier> listViews(Namespace namespace)
      Description copied from interface: ViewCatalog
      Return all the identifiers under this namespace.
      Specified by:
      listViews in interface ViewCatalog
      Parameters:
      namespace - a namespace
      Returns:
      a list of identifiers for views

    • loadView

      public View loadView(TableIdentifier identifier)
      Description copied from interface: ViewCatalog
      Load a view.
      Specified by:
      loadView in interface ViewCatalog
      Parameters:
      identifier - a view identifier
      Returns:
      instance of View implementation referred by the identifier

    • buildView

      public ViewBuilder buildView(TableIdentifier identifier)
      Description copied from interface: ViewCatalog
      Instantiate a builder to create or replace a SQL view.
      Specified by:
      buildView in interface ViewCatalog
      Parameters:
      identifier - a view identifier
      Returns:
      a view builder

    • dropView

      public boolean dropView(TableIdentifier identifier)
      Description copied from interface: ViewCatalog
      Drop a view.
      Specified by:
      dropView in interface ViewCatalog
      Parameters:
      identifier - a view identifier
      Returns:
      true if the view was dropped, false if the view did not exist

    • renameView

      public void renameView(TableIdentifier from, TableIdentifier to)
      Description copied from interface: ViewCatalog
      Rename a view.
      Specified by:
      renameView in interface ViewCatalog
      Parameters:
      from - identifier of the view to rename
      to - new view identifier

    • viewExists

      public boolean viewExists(TableIdentifier identifier)
      Description copied from interface: ViewCatalog
      Check whether view exists.
      Specified by:
      viewExists in interface ViewCatalog
      Parameters:
      identifier - a view identifier
      Returns:
      true if the view exists, false otherwise

    • invalidateView

      public void invalidateView(TableIdentifier identifier)
      Description copied from interface: ViewCatalog
      Invalidate cached view metadata from current catalog.

      If the view is already loaded or cached, drop cached data. If the view does not exist or is not cached, do nothing.

      Specified by:
      invalidateView in interface ViewCatalog
      Parameters:
      identifier - a view identifier