Package org.apache.iceberg.actions
Class RemoveOrphanFilesAction
- java.lang.Object
-
- org.apache.iceberg.actions.RemoveOrphanFilesAction
-
- All Implemented Interfaces:
Action<java.util.List<java.lang.String>>
public class RemoveOrphanFilesAction extends java.lang.Object
An action that removes orphan metadata and data files by listing a given location and comparing the actual files in that location with data and metadata files referenced by all valid snapshots. The location must be accessible for listing via the HadoopFileSystem
.By default, this action cleans up the table location returned by
Table.location()
and removes unreachable files that are older than 3 days usingTable.io()
. The behavior can be modified by passing a custom location tolocation
and a custom timestamp toolderThan(long)
. For example, someone might point this action to the data folder to clean up only orphan data files. In addition, there is a way to configure an alternative delete method viadeleteWith(Consumer)
.Note: It is dangerous to call this action with a short retention interval as it might corrupt the state of the table if another operation is writing at the same time.
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description RemoveOrphanFilesAction
deleteWith(java.util.function.Consumer<java.lang.String> newDeleteFunc)
Passes an alternative delete implementation that will be used to delete orphan files.java.util.List<java.lang.String>
execute()
Executes this action.RemoveOrphanFilesAction
location(java.lang.String newLocation)
Removes orphan files in the given location.protected java.lang.String
metadataTableName(MetadataTableType type)
RemoveOrphanFilesAction
olderThan(long newOlderThanTimestamp)
Removes orphan files that are older than the given timestamp.protected Table
table()
-
-
-
Method Detail
-
table
protected Table table()
-
location
public RemoveOrphanFilesAction location(java.lang.String newLocation)
Removes orphan files in the given location.- Parameters:
newLocation
- a location- Returns:
- this for method chaining
-
olderThan
public RemoveOrphanFilesAction olderThan(long newOlderThanTimestamp)
Removes orphan files that are older than the given timestamp.- Parameters:
newOlderThanTimestamp
- a timestamp in milliseconds- Returns:
- this for method chaining
-
deleteWith
public RemoveOrphanFilesAction deleteWith(java.util.function.Consumer<java.lang.String> newDeleteFunc)
Passes an alternative delete implementation that will be used to delete orphan files.- Parameters:
newDeleteFunc
- a delete func- Returns:
- this for method chaining
-
execute
public java.util.List<java.lang.String> execute()
Description copied from interface:Action
Executes this action.- Returns:
- the result of this action
-
metadataTableName
protected java.lang.String metadataTableName(MetadataTableType type)
-
-