Package org.apache.iceberg

Class CatalogUtil

java.lang.Object
org.apache.iceberg.CatalogUtil
public class CatalogUtil extends Object

  • Field Details

  • Method Details

    • dropTableData

      public static void dropTableData(FileIO io, TableMetadata metadata)
      Drops all data and metadata files referenced by TableMetadata.

      This should be called by dropTable implementations to clean up table files once the table has been dropped in the metastore.

      Parameters:
      io - a FileIO to use for deletes
      metadata - the last valid TableMetadata instance for a dropped table.

    • dropViewMetadata

      public static void dropViewMetadata(FileIO io, ViewMetadata metadata)
      Drops view metadata files referenced by ViewMetadata.

      This should be called by dropView implementations

      Parameters:
      io - a FileIO to use for deletes
      metadata - the last valid ViewMetadata instance for a dropped view.

    • deleteFiles

      public static void deleteFiles(FileIO io, Iterable<String> files, String type, boolean concurrent)
      Helper to delete files. Bulk deletion is used if possible.
      Parameters:
      io - FileIO for deletes
      files - files to delete
      type - type of files being deleted
      concurrent - controls concurrent deletion. Only applicable for non-bulk FileIO

    • loadCatalog

      public static Catalog loadCatalog(String impl, String catalogName, Map<String,String> properties, Object hadoopConf)
      Load a custom catalog implementation.

      The catalog must have a no-arg constructor. If the class implements Configurable, a Hadoop config will be passed using Configurable.setConf. Catalog.initialize(String catalogName, Map options) is called to complete the initialization.

      Parameters:
      impl - catalog implementation full class name
      catalogName - catalog name
      properties - catalog properties
      hadoopConf - hadoop configuration if needed
      Returns:
      initialized catalog object
      Throws:
      IllegalArgumentException - if no-arg constructor not found or error during initialization

    • buildIcebergCatalog

      public static Catalog buildIcebergCatalog(String name, Map<String,String> options, Object conf)
      Build an Iceberg Catalog based on a map of catalog properties and optional Hadoop configuration.

      This method examines both the ICEBERG_CATALOG_TYPE and CatalogProperties.CATALOG_IMPL properties to determine the catalog implementation to load. If nothing is specified for both properties, Hive catalog will be loaded by default.

      Parameters:
      name - catalog name
      options - catalog properties
      conf - a Hadoop Configuration
      Returns:
      initialized catalog

    • loadFileIO

      public static FileIO loadFileIO(String impl, Map<String,String> properties, Object hadoopConf)
      Load a custom FileIO implementation.

      The implementation must have a no-arg constructor. If the class implements Configurable, a Hadoop config will be passed using Configurable.setConf. FileIO.initialize(Map properties) is called to complete the initialization.

      Parameters:
      impl - full class name of a custom FileIO implementation
      properties - used to initialize the FileIO implementation
      hadoopConf - a hadoop Configuration
      Returns:
      FileIO class
      Throws:
      IllegalArgumentException - if class path not found or right constructor not found or the loaded class cannot be cast to the given interface type

    • loadFileIO

      public static FileIO loadFileIO(String impl, Map<String,String> properties, Object hadoopConf, List<StorageCredential> storageCredentials)
      Load a custom FileIO implementation.

      The implementation must have a no-arg constructor. If the class implements Configurable, a Hadoop config will be passed using Configurable.setConf. If the class implements SupportsStorageCredentials, the storage credentials will be passed using SupportsStorageCredentials.setCredentials(List). FileIO.initialize(Map properties) is called to complete the initialization.

      Parameters:
      impl - full class name of a custom FileIO implementation
      properties - used to initialize the FileIO implementation
      hadoopConf - a hadoop Configuration
      storageCredentials - the storage credentials to configure if the FileIO implementation implements SupportsStorageCredentials
      Returns:
      FileIO class
      Throws:
      IllegalArgumentException - if class path not found or right constructor not found or the loaded class cannot be cast to the given interface type

    • configureHadoopConf

      public static void configureHadoopConf(Object maybeConfigurable, Object conf)
      Dynamically detects whether an object is a Hadoop Configurable and calls setConf.
      Parameters:
      maybeConfigurable - an object that may be Configurable
      conf - a Configuration

    • loadMetricsReporter

      public static MetricsReporter loadMetricsReporter(Map<String,String> properties)
      Load a custom MetricsReporter implementation.

      The implementation must have a no-arg constructor.

      Parameters:
      properties - catalog properties which contains class name of a custom MetricsReporter implementation
      Returns:
      An initialized MetricsReporter.
      Throws:
      IllegalArgumentException - if class path not found or right constructor not found or the loaded class cannot be cast to the given interface type

    • fullTableName

      public static String fullTableName(String catalogName, TableIdentifier identifier)

    • deleteRemovedMetadataFiles

      public static void deleteRemovedMetadataFiles(FileIO io, TableMetadata base, TableMetadata metadata)
      Deletes the oldest metadata files if TableProperties.METADATA_DELETE_AFTER_COMMIT_ENABLED is true.
      Parameters:
      io - FileIO instance to use for deletes
      base - table metadata on which previous versions were based
      metadata - new table metadata with updated previous versions