Package org.apache.iceberg.flink.maintenance.operator

Class DataFileRewriteRunner

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.ProcessFunction<DataFileRewritePlanner.PlannedGroup,DataFileRewriteRunner.ExecutedGroup>

org.apache.iceberg.flink.maintenance.operator.DataFileRewriteRunner

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction

@Internal
public class DataFileRewriteRunner
extends org.apache.flink.streaming.api.functions.ProcessFunction<DataFileRewritePlanner.PlannedGroup,DataFileRewriteRunner.ExecutedGroup>

Executes a rewrite for a single DataFileRewritePlanner.PlannedGroup. Reads the files with the standard FileScanTaskReader, so the delete files are considered, and writes using the TaskWriterFactory. The output is an DataFileRewriteRunner.ExecutedGroup.

See Also:

• Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DataFileRewriteRunner.ExecutedGroup

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.ProcessFunction

org.apache.flink.streaming.api.functions.ProcessFunction.Context, org.apache.flink.streaming.api.functions.ProcessFunction.OnTimerContext

Constructor Summary

Constructors

Constructor	Description
DataFileRewriteRunner(String tableName, String taskName, int taskIndex)

Method Summary

Modifier and Type	Method	Description
void	open(org.apache.flink.api.common.functions.OpenContext context)
void	processElement(DataFileRewritePlanner.PlannedGroup value, org.apache.flink.streaming.api.functions.ProcessFunction<DataFileRewritePlanner.PlannedGroup,DataFileRewriteRunner.ExecutedGroup>.org.apache.flink.streaming.api.functions.ProcessFunction.Context ctx, org.apache.flink.util.Collector<DataFileRewriteRunner.ExecutedGroup> out)

Methods inherited from class org.apache.flink.streaming.api.functions.ProcessFunction

onTimer

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

close, getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DataFileRewriteRunner

public DataFileRewriteRunner(String tableName,
 String taskName,
 int taskIndex)

Method Details

open

public void open(org.apache.flink.api.common.functions.OpenContext context)

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

processElement

public void processElement(DataFileRewritePlanner.PlannedGroup value,
 org.apache.flink.streaming.api.functions.ProcessFunction<DataFileRewritePlanner.PlannedGroup,DataFileRewriteRunner.ExecutedGroup>.org.apache.flink.streaming.api.functions.ProcessFunction.Context ctx,
 org.apache.flink.util.Collector<DataFileRewriteRunner.ExecutedGroup> out)
                    throws Exception

Specified by:

processElement in class org.apache.flink.streaming.api.functions.ProcessFunction<DataFileRewritePlanner.PlannedGroup,DataFileRewriteRunner.ExecutedGroup>

Throws:

Exception