Package org.apache.iceberg.rest

Class RESTCatalog

    • Constructor Detail

      • RESTCatalog

        public RESTCatalog()

      • RESTCatalog

        public RESTCatalog​(java.util.function.Function<java.util.Map<java.lang.String,​java.lang.String>,​RESTClient> clientBuilder)

    • Method Detail

      • initialize

        public void initialize​(java.lang.String name,
                       java.util.Map<java.lang.String,​java.lang.String> props)
        Description copied from interface: Catalog
        Initialize a catalog given a custom name and a map of catalog properties.

        A custom Catalog implementation must have a no-arg constructor. A compute engine like Spark or Flink will first initialize the catalog without any arguments, and then call this method to complete catalog initialization with properties passed into the engine.

        Specified by:
        initialize in interface Catalog
        Specified by:
        initialize in interface ViewCatalog
        Parameters:
        name - a custom name for the catalog
        props - catalog properties

      • name

        public java.lang.String name()
        Description copied from interface: Catalog
        Return the name for this catalog.
        Specified by:
        name in interface Catalog
        Specified by:
        name in interface ViewCatalog
        Returns:
        this catalog's name

      • properties

        public java.util.Map<java.lang.String,​java.lang.String> properties()

      • listTables

        public java.util.List<TableIdentifier> listTables​(Namespace ns)
        Description copied from interface: Catalog
        Return all the identifiers under this namespace.
        Specified by:
        listTables in interface Catalog
        Parameters:
        ns - a namespace
        Returns:
        a list of identifiers for tables

      • tableExists

        public boolean tableExists​(TableIdentifier ident)
        Description copied from interface: Catalog
        Check whether table exists.
        Specified by:
        tableExists in interface Catalog
        Parameters:
        ident - a table identifier
        Returns:
        true if the table exists, false otherwise

      • loadTable

        public Table loadTable​(TableIdentifier ident)
        Description copied from interface: Catalog
        Load a table.
        Specified by:
        loadTable in interface Catalog
        Parameters:
        ident - a table identifier
        Returns:
        instance of Table implementation referred by tableIdentifier

      • invalidateTable

        public void invalidateTable​(TableIdentifier ident)
        Description copied from interface: Catalog
        Invalidate cached table metadata from current catalog.

        If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing.

        Specified by:
        invalidateTable in interface Catalog
        Parameters:
        ident - a table identifier

      • buildTable

        public Catalog.TableBuilder buildTable​(TableIdentifier ident,
                                       Schema schema)
        Description copied from interface: Catalog
        /** Instantiate a builder to either create a table or start a create/replace transaction.
        Specified by:
        buildTable in interface Catalog
        Parameters:
        ident - a table identifier
        schema - a schema
        Returns:
        the builder to create a table or start a create/replace transaction

      • createTable

        public Table createTable​(TableIdentifier ident,
                         Schema schema,
                         PartitionSpec spec,
                         java.lang.String location,
                         java.util.Map<java.lang.String,​java.lang.String> props)
        Description copied from interface: Catalog
        Create a table.
        Specified by:
        createTable in interface Catalog
        Parameters:
        ident - a table identifier
        schema - a schema
        spec - a partition spec
        location - a location for the table; leave null if unspecified
        props - a string map of table properties
        Returns:
        a Table instance

      • createTable

        public Table createTable​(TableIdentifier ident,
                         Schema schema,
                         PartitionSpec spec,
                         java.util.Map<java.lang.String,​java.lang.String> props)
        Description copied from interface: Catalog
        Create a table.
        Specified by:
        createTable in interface Catalog
        Parameters:
        ident - a table identifier
        schema - a schema
        spec - a partition spec
        props - a string map of table properties
        Returns:
        a Table instance

      • createTable

        public Table createTable​(TableIdentifier ident,
                         Schema schema,
                         PartitionSpec spec)
        Description copied from interface: Catalog
        Create a table.
        Specified by:
        createTable in interface Catalog
        Parameters:
        ident - a table identifier
        schema - a schema
        spec - a partition spec
        Returns:
        a Table instance

      • createTable

        public Table createTable​(TableIdentifier identifier,
                         Schema schema)
        Description copied from interface: Catalog
        Create an unpartitioned table.
        Specified by:
        createTable in interface Catalog
        Parameters:
        identifier - a table identifier
        schema - a schema
        Returns:
        a Table instance

      • newCreateTableTransaction

        public Transaction newCreateTableTransaction​(TableIdentifier ident,
                                             Schema schema,
                                             PartitionSpec spec,
                                             java.lang.String location,
                                             java.util.Map<java.lang.String,​java.lang.String> props)
        Description copied from interface: Catalog
        Start a transaction to create a table.
        Specified by:
        newCreateTableTransaction in interface Catalog
        Parameters:
        ident - a table identifier
        schema - a schema
        spec - a partition spec
        location - a location for the table; leave null if unspecified
        props - a string map of table properties
        Returns:
        a Transaction to create the table

      • newCreateTableTransaction

        public Transaction newCreateTableTransaction​(TableIdentifier ident,
                                             Schema schema,
                                             PartitionSpec spec,
                                             java.util.Map<java.lang.String,​java.lang.String> props)
        Description copied from interface: Catalog
        Start a transaction to create a table.
        Specified by:
        newCreateTableTransaction in interface Catalog
        Parameters:
        ident - a table identifier
        schema - a schema
        spec - a partition spec
        props - a string map of table properties
        Returns:
        a Transaction to create the table

      • newReplaceTableTransaction

        public Transaction newReplaceTableTransaction​(TableIdentifier ident,
                                              Schema schema,
                                              PartitionSpec spec,
                                              java.lang.String location,
                                              java.util.Map<java.lang.String,​java.lang.String> props,
                                              boolean orCreate)
        Description copied from interface: Catalog
        Start a transaction to replace a table.
        Specified by:
        newReplaceTableTransaction in interface Catalog
        Parameters:
        ident - a table identifier
        schema - a schema
        spec - a partition spec
        location - a location for the table; leave null if unspecified
        props - a string map of table properties
        orCreate - whether to create the table if not exists
        Returns:
        a Transaction to replace the table

      • newReplaceTableTransaction

        public Transaction newReplaceTableTransaction​(TableIdentifier ident,
                                              Schema schema,
                                              PartitionSpec spec,
                                              java.util.Map<java.lang.String,​java.lang.String> props,
                                              boolean orCreate)
        Description copied from interface: Catalog
        Start a transaction to replace a table.
        Specified by:
        newReplaceTableTransaction in interface Catalog
        Parameters:
        ident - a table identifier
        schema - a schema
        spec - a partition spec
        props - a string map of table properties
        orCreate - whether to create the table if not exists
        Returns:
        a Transaction to replace the table

      • newReplaceTableTransaction

        public Transaction newReplaceTableTransaction​(TableIdentifier ident,
                                              Schema schema,
                                              PartitionSpec spec,
                                              boolean orCreate)
        Description copied from interface: Catalog
        Start a transaction to replace a table.
        Specified by:
        newReplaceTableTransaction in interface Catalog
        Parameters:
        ident - a table identifier
        schema - a schema
        spec - a partition spec
        orCreate - whether to create the table if not exists
        Returns:
        a Transaction to replace the table

      • newReplaceTableTransaction

        public Transaction newReplaceTableTransaction​(TableIdentifier ident,
                                              Schema schema,
                                              boolean orCreate)
        Description copied from interface: Catalog
        Start a transaction to replace a table.
        Specified by:
        newReplaceTableTransaction in interface Catalog
        Parameters:
        ident - a table identifier
        schema - a schema
        orCreate - whether to create the table if not exists
        Returns:
        a Transaction to replace the table

      • dropTable

        public boolean dropTable​(TableIdentifier ident)
        Description copied from interface: Catalog
        Drop a table and delete all data and metadata files.
        Specified by:
        dropTable in interface Catalog
        Parameters:
        ident - a table identifier
        Returns:
        true if the table was dropped, false if the table did not exist

      • dropTable

        public boolean dropTable​(TableIdentifier ident,
                         boolean purge)
        Description copied from interface: Catalog
        Drop a table; optionally delete data and metadata files.

        If purge is set to true the implementation should delete all data and metadata files.

        Specified by:
        dropTable in interface Catalog
        Parameters:
        ident - a table identifier
        purge - if true, delete all data and metadata files in the table
        Returns:
        true if the table was dropped, false if the table did not exist

      • registerTable

        public Table registerTable​(TableIdentifier ident,
                           java.lang.String metadataFileLocation)
        Description copied from interface: Catalog
        Register a table with the catalog if it does not exist.
        Specified by:
        registerTable in interface Catalog
        Parameters:
        ident - a table identifier
        metadataFileLocation - the location of a metadata file
        Returns:
        a Table instance

      • createNamespace

        public void createNamespace​(Namespace ns,
                            java.util.Map<java.lang.String,​java.lang.String> props)
        Description copied from interface: SupportsNamespaces
        Create a namespace in the catalog.
        Specified by:
        createNamespace in interface SupportsNamespaces
        Parameters:
        ns - a multi-part namespace
        props - a string Map of properties for the given namespace

      • listNamespaces

        public java.util.List<Namespace> listNamespaces​(Namespace ns)
                                         throws NoSuchNamespaceException
        Description copied from interface: SupportsNamespaces
        List child namespaces from the namespace.

        For two existing tables named 'a.b.c.table' and 'a.b.d.table', this method returns:

        • Given: Namespace.empty()
        • Returns: Namespace.of("a")
        • Given: Namespace.of("a")
        • Returns: Namespace.of("a", "b")
        • Given: Namespace.of("a", "b")
        • Returns: Namespace.of("a", "b", "c") and Namespace.of("a", "b", "d")
        • Given: Namespace.of("a", "b", "c")
        • Returns: empty list, because there are no child namespaces
        Specified by:
        listNamespaces in interface SupportsNamespaces
        Returns:
        a List of child Namespace names from the given namespace
        Throws:
        NoSuchNamespaceException - If the namespace does not exist (optional)

      • setProperties

        public boolean setProperties​(Namespace ns,
                             java.util.Map<java.lang.String,​java.lang.String> props)
                      throws NoSuchNamespaceException
        Description copied from interface: SupportsNamespaces
        Set a collection of properties on a namespace in the catalog.

        Properties that are not in the given map are not modified or removed by this method.

        Specified by:
        setProperties in interface SupportsNamespaces
        Parameters:
        ns - a namespace. Namespace
        props - a collection of metadata to apply to the namespace
        Throws:
        NoSuchNamespaceException - If the namespace does not exist (optional)

      • setConf

        public void setConf​(java.lang.Object conf)
        Specified by:
        setConf in interface Configurable<java.lang.Object>

      • close

        public void close()
           throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Throws:
        java.io.IOException

      • commitTransaction

        public void commitTransaction​(java.util.List<TableCommit> commits)

      • commitTransaction

        public void commitTransaction​(TableCommit... commits)

      • listViews

        public java.util.List<TableIdentifier> listViews​(Namespace namespace)
        Description copied from interface: ViewCatalog
        Return all the identifiers under this namespace.
        Specified by:
        listViews in interface ViewCatalog
        Parameters:
        namespace - a namespace
        Returns:
        a list of identifiers for views

      • loadView

        public View loadView​(TableIdentifier identifier)
        Description copied from interface: ViewCatalog
        Load a view.
        Specified by:
        loadView in interface ViewCatalog
        Parameters:
        identifier - a view identifier
        Returns:
        instance of View implementation referred by the identifier

      • buildView

        public ViewBuilder buildView​(TableIdentifier identifier)
        Description copied from interface: ViewCatalog
        Instantiate a builder to create or replace a SQL view.
        Specified by:
        buildView in interface ViewCatalog
        Parameters:
        identifier - a view identifier
        Returns:
        a view builder

      • dropView

        public boolean dropView​(TableIdentifier identifier)
        Description copied from interface: ViewCatalog
        Drop a view.
        Specified by:
        dropView in interface ViewCatalog
        Parameters:
        identifier - a view identifier
        Returns:
        true if the view was dropped, false if the view did not exist

      • viewExists

        public boolean viewExists​(TableIdentifier identifier)
        Description copied from interface: ViewCatalog
        Check whether view exists.
        Specified by:
        viewExists in interface ViewCatalog
        Parameters:
        identifier - a view identifier
        Returns:
        true if the view exists, false otherwise

      • invalidateView

        public void invalidateView​(TableIdentifier identifier)
        Description copied from interface: ViewCatalog
        Invalidate cached view metadata from current catalog.

        If the view is already loaded or cached, drop cached data. If the view does not exist or is not cached, do nothing.

        Specified by:
        invalidateView in interface ViewCatalog
        Parameters:
        identifier - a view identifier