org.apache.iceberg.spark

Class FileRewriteCoordinator

java.lang.Object

org.apache.iceberg.spark.FileRewriteCoordinator

public class FileRewriteCoordinator
extends java.lang.Object

Method Summary

Modifier and Type	Method and Description
void	clearRewrite(Table table, java.lang.String fileSetID)
java.util.Set<DataFile>	fetchNewDataFiles(Table table, java.lang.String fileSetID)
java.util.Set<java.lang.String>	fetchSetIDs(Table table)
static FileRewriteCoordinator	get()
void	stageRewrite(Table table, java.lang.String fileSetID, java.util.Set<DataFile> newDataFiles) Called to persist the output of a rewrite action for a specific group.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

get

public static FileRewriteCoordinator get()

stageRewrite

public void stageRewrite(Table table,
                         java.lang.String fileSetID,
                         java.util.Set<DataFile> newDataFiles)

Called to persist the output of a rewrite action for a specific group. Since the write is done via a Spark Datasource, we have to propagate the result through this side-effect call.

Parameters:

table - table where the rewrite is occurring

fileSetID - the id used to identify the source set of files being rewritten

newDataFiles - the new files which have been written

fetchNewDataFiles

public java.util.Set<DataFile> fetchNewDataFiles(Table table,
                                                 java.lang.String fileSetID)

clearRewrite

public void clearRewrite(Table table,
                         java.lang.String fileSetID)

fetchSetIDs

public java.util.Set<java.lang.String> fetchSetIDs(Table table)