org.apache.iceberg.spark.procedures

Class RemoveOrphanFilesProcedure

java.lang.Object

org.apache.iceberg.spark.procedures.RemoveOrphanFilesProcedure

All Implemented Interfaces:

Procedure

public class RemoveOrphanFilesProcedure
extends java.lang.Object

A procedure that removes orphan files in a table.

See Also:

Actions.removeOrphanFiles()

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	org.apache.iceberg.spark.procedures.BaseProcedure.Builder<T extends org.apache.iceberg.spark.procedures.BaseProcedure>

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.apache.spark.sql.types.DataType	STRING_MAP

Method Summary

Modifier and Type	Method and Description
static SparkProcedures.ProcedureBuilder	builder()
org.apache.spark.sql.catalyst.InternalRow[]	call(org.apache.spark.sql.catalyst.InternalRow args) Executes this procedure.
java.lang.String	description() Returns the description of this procedure.
protected SparkTable	loadSparkTable(org.apache.spark.sql.connector.catalog.Identifier ident)
protected <T> T	modifyIcebergTable(org.apache.spark.sql.connector.catalog.Identifier ident, java.util.function.Function<Table,T> func)
protected org.apache.spark.sql.catalyst.InternalRow	newInternalRow(java.lang.Object... values)
org.apache.spark.sql.types.StructType	outputType() Returns the type of rows produced by this procedure.
ProcedureParameter[]	parameters() Returns the input parameters of this procedure.
protected void	refreshSparkCache(org.apache.spark.sql.connector.catalog.Identifier ident, org.apache.spark.sql.connector.catalog.Table table)
protected org.apache.spark.sql.SparkSession	spark()
protected org.apache.spark.sql.connector.catalog.TableCatalog	tableCatalog()
protected Spark3Util.CatalogAndIdentifier	toCatalogAndIdentifier(java.lang.String identifierAsString, java.lang.String argName, org.apache.spark.sql.connector.catalog.CatalogPlugin catalog)
protected org.apache.spark.sql.connector.catalog.Identifier	toIdentifier(java.lang.String identifierAsString, java.lang.String argName)
protected <T> T	withIcebergTable(org.apache.spark.sql.connector.catalog.Identifier ident, java.util.function.Function<Table,T> func)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STRING_MAP

protected static final org.apache.spark.sql.types.DataType STRING_MAP

Method Detail

builder

public static SparkProcedures.ProcedureBuilder builder()

parameters

public ProcedureParameter[] parameters()

Description copied from interface: Procedure

Returns the input parameters of this procedure.

outputType

public org.apache.spark.sql.types.StructType outputType()

Description copied from interface: Procedure

Returns the type of rows produced by this procedure.

call

public org.apache.spark.sql.catalyst.InternalRow[] call(org.apache.spark.sql.catalyst.InternalRow args)

Description copied from interface: Procedure

Executes this procedure.

Spark will align the provided arguments according to the input parameters defined in Procedure.parameters() either by position or by name before execution.

Implementations may provide a summary of execution by returning one or many rows as a result. The schema of output rows must match the defined output type in Procedure.outputType().

Parameters:

args - input arguments

Returns:

the result of executing this procedure with the given arguments

description

public java.lang.String description()

Description copied from interface: Procedure

Returns the description of this procedure.

spark

protected org.apache.spark.sql.SparkSession spark()

tableCatalog

protected org.apache.spark.sql.connector.catalog.TableCatalog tableCatalog()

modifyIcebergTable

protected <T> T modifyIcebergTable(org.apache.spark.sql.connector.catalog.Identifier ident,
                                   java.util.function.Function<Table,T> func)

withIcebergTable

protected <T> T withIcebergTable(org.apache.spark.sql.connector.catalog.Identifier ident,
                                 java.util.function.Function<Table,T> func)

toIdentifier

protected org.apache.spark.sql.connector.catalog.Identifier toIdentifier(java.lang.String identifierAsString,
                                                                         java.lang.String argName)

toCatalogAndIdentifier

protected Spark3Util.CatalogAndIdentifier toCatalogAndIdentifier(java.lang.String identifierAsString,
                                                                 java.lang.String argName,
                                                                 org.apache.spark.sql.connector.catalog.CatalogPlugin catalog)

loadSparkTable

protected SparkTable loadSparkTable(org.apache.spark.sql.connector.catalog.Identifier ident)

refreshSparkCache

protected void refreshSparkCache(org.apache.spark.sql.connector.catalog.Identifier ident,
                                 org.apache.spark.sql.connector.catalog.Table table)

newInternalRow

protected org.apache.spark.sql.catalyst.InternalRow newInternalRow(java.lang.Object... values)