Package org.apache.iceberg

Class BaseMetastoreTableOperations

java.lang.Object
org.apache.iceberg.BaseMetastoreOperations
org.apache.iceberg.BaseMetastoreTableOperations
All Implemented Interfaces:
TableOperations
Direct Known Subclasses:
EcsTableOperations, HiveTableOperations, NessieTableOperations
public abstract class BaseMetastoreTableOperations extends BaseMetastoreOperations implements TableOperations

  • Field Details

  • Constructor Details

    • BaseMetastoreTableOperations

      protected BaseMetastoreTableOperations()

  • Method Details

    • tableName

      protected abstract String tableName()
      The full name of the table used for logging purposes only. For example for HiveTableOperations it is catalogName + "." + database + "." + table.
      Returns:
      The full name

    • current

      public TableMetadata current()
      Description copied from interface: TableOperations
      Return the currently loaded table metadata, without checking for updates.
      Specified by:
      current in interface TableOperations
      Returns:
      table metadata

    • currentMetadataLocation

      public String currentMetadataLocation()

    • currentVersion

      public int currentVersion()

    • refresh

      public TableMetadata refresh()
      Description copied from interface: TableOperations
      Return the current table metadata after checking for updates.
      Specified by:
      refresh in interface TableOperations
      Returns:
      table metadata

    • doRefresh

      protected void doRefresh()

    • commit

      public void commit(TableMetadata base, TableMetadata metadata)
      Description copied from interface: TableOperations
      Replace the base table metadata with a new version.

      This method should implement and document atomicity guarantees.

      Implementations must check that the base metadata is current to avoid overwriting updates. Once the atomic commit operation succeeds, implementations must not perform any operations that may fail because failure in this method cannot be distinguished from commit failure.

      Implementations must throw a CommitStateUnknownException in cases where it cannot be determined if the commit succeeded or failed. For example if a network partition causes the confirmation of the commit to be lost, the implementation should throw a CommitStateUnknownException. This is important because downstream users of this API need to know whether they can clean up the commit or not, if the state is unknown then it is not safe to remove any files. All other exceptions will be treated as if the commit has failed.

      Specified by:
      commit in interface TableOperations
      Parameters:
      base - table metadata on which changes were based
      metadata - new table metadata with updates

    • doCommit

      protected void doCommit(TableMetadata base, TableMetadata metadata)

    • requestRefresh

      protected void requestRefresh()

    • disableRefresh

      protected void disableRefresh()

    • writeNewMetadataIfRequired

      protected String writeNewMetadataIfRequired(boolean newTable, TableMetadata metadata)

    • writeNewMetadata

      protected String writeNewMetadata(TableMetadata metadata, int newVersion)

    • refreshFromMetadataLocation

      protected void refreshFromMetadataLocation(String newLocation)

    • refreshFromMetadataLocation

      protected void refreshFromMetadataLocation(String newLocation, int numRetries)

    • refreshFromMetadataLocation

      protected void refreshFromMetadataLocation(String newLocation, Predicate<Exception> shouldRetry, int numRetries)

    • refreshFromMetadataLocation

      protected void refreshFromMetadataLocation(String newLocation, Predicate<Exception> shouldRetry, int numRetries, Function<String,TableMetadata> metadataLoader)

    • metadataFileLocation

      public String metadataFileLocation(String filename)
      Description copied from interface: TableOperations
      Given the name of a metadata file, obtain the full path of that file using an appropriate base location of the implementation's choosing.

      The file may not exist yet, in which case the path should be returned as if it were to be created by e.g. FileIO.newOutputFile(String).

      Specified by:
      metadataFileLocation in interface TableOperations

    • locationProvider

      public LocationProvider locationProvider()
      Description copied from interface: TableOperations
      Returns a LocationProvider that supplies locations for new new data files.
      Specified by:
      locationProvider in interface TableOperations
      Returns:
      a location provider configured for the current table state

    • temp

      public TableOperations temp(TableMetadata uncommittedMetadata)
      Description copied from interface: TableOperations
      Return a temporary TableOperations instance that uses configuration from uncommitted metadata.

      This is called by transactions when uncommitted table metadata should be used; for example, to create a metadata file location based on metadata in the transaction that has not been committed.

      Transactions will not call TableOperations.refresh() or TableOperations.commit(TableMetadata, TableMetadata).

      Specified by:
      temp in interface TableOperations
      Parameters:
      uncommittedMetadata - uncommitted table metadata
      Returns:
      a temporary table operations that behaves like the uncommitted metadata is current

    • checkCommitStatus

      protected BaseMetastoreOperations.CommitStatus checkCommitStatus(String newMetadataLocation, TableMetadata config)
      Attempt to load the table and see if any current or past metadata location matches the one we were attempting to set. This is used as a last resort when we are dealing with exceptions that may indicate the commit has failed but are not proof that this is the case. Past locations must also be searched on the chance that a second committer was able to successfully commit on top of our commit. When the newMetadataLocation is not found, the method returns BaseMetastoreOperations.CommitStatus.UNKNOWN.
      Parameters:
      newMetadataLocation - the path of the new commit file
      config - metadata to use for configuration
      Returns:
      Commit Status of Success, Unknown

    • checkCommitStatusStrict

      protected BaseMetastoreOperations.CommitStatus checkCommitStatusStrict(String newMetadataLocation, TableMetadata config)
      Attempt to load the table and see if any current or past metadata location matches the one we were attempting to set. This is used as a last resort when we are dealing with exceptions that may indicate the commit has failed but are not proof that this is the case. Past locations must also be searched on the chance that a second committer was able to successfully commit on top of our commit. When the newMetadataLocation is not found, the method returns BaseMetastoreOperations.CommitStatus.FAILURE.
      Parameters:
      newMetadataLocation - the path of the new commit file
      config - metadata to use for configuration
      Returns:
      Commit Status of Success, Failure or Unknown