Class BaseMetastoreTableOperations
- java.lang.Object
-
- org.apache.iceberg.BaseMetastoreTableOperations
-
- All Implemented Interfaces:
TableOperations
- Direct Known Subclasses:
EcsTableOperations
,HiveTableOperations
,NessieTableOperations
public abstract class BaseMetastoreTableOperations extends java.lang.Object implements TableOperations
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static class
BaseMetastoreTableOperations.CommitStatus
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
ICEBERG_TABLE_TYPE_VALUE
static java.lang.String
METADATA_LOCATION_PROP
static java.lang.String
PREVIOUS_METADATA_LOCATION_PROP
static java.lang.String
TABLE_TYPE_PROP
-
Constructor Summary
Constructors Modifier Constructor Description protected
BaseMetastoreTableOperations()
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected BaseMetastoreTableOperations.CommitStatus
checkCommitStatus(java.lang.String newMetadataLocation, TableMetadata config)
Attempt to load the table and see if any current or past metadata location matches the one we were attempting to set.void
commit(TableMetadata base, TableMetadata metadata)
Replace the base table metadata with a new version.TableMetadata
current()
Return the currently loaded table metadata, without checking for updates.java.lang.String
currentMetadataLocation()
int
currentVersion()
protected void
disableRefresh()
protected void
doCommit(TableMetadata base, TableMetadata metadata)
protected void
doRefresh()
LocationProvider
locationProvider()
Returns aLocationProvider
that supplies locations for new new data files.java.lang.String
metadataFileLocation(java.lang.String filename)
Given the name of a metadata file, obtain the full path of that file using an appropriate base location of the implementation's choosing.TableMetadata
refresh()
Return the current table metadata after checking for updates.protected void
refreshFromMetadataLocation(java.lang.String newLocation)
protected void
refreshFromMetadataLocation(java.lang.String newLocation, int numRetries)
protected void
refreshFromMetadataLocation(java.lang.String newLocation, java.util.function.Predicate<java.lang.Exception> shouldRetry, int numRetries)
protected void
refreshFromMetadataLocation(java.lang.String newLocation, java.util.function.Predicate<java.lang.Exception> shouldRetry, int numRetries, java.util.function.Function<java.lang.String,TableMetadata> metadataLoader)
protected void
requestRefresh()
protected abstract java.lang.String
tableName()
The full name of the table used for logging purposes only.TableOperations
temp(TableMetadata uncommittedMetadata)
Return a temporaryTableOperations
instance that uses configuration from uncommitted metadata.protected java.lang.String
writeNewMetadata(TableMetadata metadata, int newVersion)
protected java.lang.String
writeNewMetadataIfRequired(boolean newTable, TableMetadata metadata)
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.iceberg.TableOperations
encryption, io, newSnapshotId
-
-
-
-
Field Detail
-
TABLE_TYPE_PROP
public static final java.lang.String TABLE_TYPE_PROP
- See Also:
- Constant Field Values
-
ICEBERG_TABLE_TYPE_VALUE
public static final java.lang.String ICEBERG_TABLE_TYPE_VALUE
- See Also:
- Constant Field Values
-
METADATA_LOCATION_PROP
public static final java.lang.String METADATA_LOCATION_PROP
- See Also:
- Constant Field Values
-
PREVIOUS_METADATA_LOCATION_PROP
public static final java.lang.String PREVIOUS_METADATA_LOCATION_PROP
- See Also:
- Constant Field Values
-
-
Method Detail
-
tableName
protected abstract java.lang.String tableName()
The full name of the table used for logging purposes only. For example for HiveTableOperations it is catalogName + "." + database + "." + table.- Returns:
- The full name
-
current
public TableMetadata current()
Description copied from interface:TableOperations
Return the currently loaded table metadata, without checking for updates.- Specified by:
current
in interfaceTableOperations
- Returns:
- table metadata
-
currentMetadataLocation
public java.lang.String currentMetadataLocation()
-
currentVersion
public int currentVersion()
-
refresh
public TableMetadata refresh()
Description copied from interface:TableOperations
Return the current table metadata after checking for updates.- Specified by:
refresh
in interfaceTableOperations
- Returns:
- table metadata
-
doRefresh
protected void doRefresh()
-
commit
public void commit(TableMetadata base, TableMetadata metadata)
Description copied from interface:TableOperations
Replace the base table metadata with a new version.This method should implement and document atomicity guarantees.
Implementations must check that the base metadata is current to avoid overwriting updates. Once the atomic commit operation succeeds, implementations must not perform any operations that may fail because failure in this method cannot be distinguished from commit failure.
Implementations must throw a
CommitStateUnknownException
in cases where it cannot be determined if the commit succeeded or failed. For example if a network partition causes the confirmation of the commit to be lost, the implementation should throw a CommitStateUnknownException. This is important because downstream users of this API need to know whether they can clean up the commit or not, if the state is unknown then it is not safe to remove any files. All other exceptions will be treated as if the commit has failed.- Specified by:
commit
in interfaceTableOperations
- Parameters:
base
- table metadata on which changes were basedmetadata
- new table metadata with updates
-
doCommit
protected void doCommit(TableMetadata base, TableMetadata metadata)
-
requestRefresh
protected void requestRefresh()
-
disableRefresh
protected void disableRefresh()
-
writeNewMetadataIfRequired
protected java.lang.String writeNewMetadataIfRequired(boolean newTable, TableMetadata metadata)
-
writeNewMetadata
protected java.lang.String writeNewMetadata(TableMetadata metadata, int newVersion)
-
refreshFromMetadataLocation
protected void refreshFromMetadataLocation(java.lang.String newLocation)
-
refreshFromMetadataLocation
protected void refreshFromMetadataLocation(java.lang.String newLocation, int numRetries)
-
refreshFromMetadataLocation
protected void refreshFromMetadataLocation(java.lang.String newLocation, java.util.function.Predicate<java.lang.Exception> shouldRetry, int numRetries)
-
refreshFromMetadataLocation
protected void refreshFromMetadataLocation(java.lang.String newLocation, java.util.function.Predicate<java.lang.Exception> shouldRetry, int numRetries, java.util.function.Function<java.lang.String,TableMetadata> metadataLoader)
-
metadataFileLocation
public java.lang.String metadataFileLocation(java.lang.String filename)
Description copied from interface:TableOperations
Given the name of a metadata file, obtain the full path of that file using an appropriate base location of the implementation's choosing.The file may not exist yet, in which case the path should be returned as if it were to be created by e.g.
FileIO.newOutputFile(String)
.- Specified by:
metadataFileLocation
in interfaceTableOperations
-
locationProvider
public LocationProvider locationProvider()
Description copied from interface:TableOperations
Returns aLocationProvider
that supplies locations for new new data files.- Specified by:
locationProvider
in interfaceTableOperations
- Returns:
- a location provider configured for the current table state
-
temp
public TableOperations temp(TableMetadata uncommittedMetadata)
Description copied from interface:TableOperations
Return a temporaryTableOperations
instance that uses configuration from uncommitted metadata.This is called by transactions when uncommitted table metadata should be used; for example, to create a metadata file location based on metadata in the transaction that has not been committed.
Transactions will not call
TableOperations.refresh()
orTableOperations.commit(TableMetadata, TableMetadata)
.- Specified by:
temp
in interfaceTableOperations
- Parameters:
uncommittedMetadata
- uncommitted table metadata- Returns:
- a temporary table operations that behaves like the uncommitted metadata is current
-
checkCommitStatus
protected BaseMetastoreTableOperations.CommitStatus checkCommitStatus(java.lang.String newMetadataLocation, TableMetadata config)
Attempt to load the table and see if any current or past metadata location matches the one we were attempting to set. This is used as a last resort when we are dealing with exceptions that may indicate the commit has failed but are not proof that this is the case. Past locations must also be searched on the chance that a second committer was able to successfully commit on top of our commit.- Parameters:
newMetadataLocation
- the path of the new commit fileconfig
- metadata to use for configuration- Returns:
- Commit Status of Success, Failure or Unknown
-
-