org.apache.iceberg.actions

Class RemoveOrphanFilesAction

java.lang.Object

org.apache.iceberg.actions.RemoveOrphanFilesAction

All Implemented Interfaces:

Action<java.util.List<java.lang.String>>

public class RemoveOrphanFilesAction
extends java.lang.Object

An action that removes orphan metadata and data files by listing a given location and comparing the actual files in that location with data and metadata files referenced by all valid snapshots. The location must be accessible for listing via the Hadoop FileSystem.

By default, this action cleans up the table location returned by Table.location() and removes unreachable files that are older than 3 days using Table.io(). The behavior can be modified by passing a custom location to location and a custom timestamp to olderThan(long). For example, someone might point this action to the data folder to clean up only orphan data files. In addition, there is a way to configure an alternative delete method via deleteWith(Consumer).

Note: It is dangerous to call this action with a short retention interval as it might corrupt the state of the table if another operation is writing at the same time.

Method Summary

Modifier and Type	Method and Description
RemoveOrphanFilesAction	deleteWith(java.util.function.Consumer<java.lang.String> newDeleteFunc) Passes an alternative delete implementation that will be used to delete orphan files.
java.util.List<java.lang.String>	execute() Executes this action.
RemoveOrphanFilesAction	location(java.lang.String newLocation) Removes orphan files in the given location.
protected java.lang.String	metadataTableName(MetadataTableType type)
RemoveOrphanFilesAction	olderThan(long newOlderThanTimestamp) Removes orphan files that are older than the given timestamp.
protected Table	table()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

table

protected Table table()

location

public RemoveOrphanFilesAction location(java.lang.String newLocation)

Removes orphan files in the given location.

Parameters:

newLocation - a location

Returns:

this for method chaining

olderThan

public RemoveOrphanFilesAction olderThan(long newOlderThanTimestamp)

Removes orphan files that are older than the given timestamp.

Parameters:

newOlderThanTimestamp - a timestamp in milliseconds

Returns:

this for method chaining

deleteWith

public RemoveOrphanFilesAction deleteWith(java.util.function.Consumer<java.lang.String> newDeleteFunc)

Passes an alternative delete implementation that will be used to delete orphan files.

Parameters:

newDeleteFunc - a delete func

Returns:

this for method chaining

execute

public java.util.List<java.lang.String> execute()

Description copied from interface: Action

Executes this action.

Returns:

the result of this action

metadataTableName

protected java.lang.String metadataTableName(MetadataTableType type)