Package org.apache.iceberg.spark
Class FileRewriteCoordinator
- java.lang.Object
-
- org.apache.iceberg.spark.FileRewriteCoordinator
-
public class FileRewriteCoordinator extends java.lang.Object
-
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
clearRewrite(Table table, java.lang.String fileSetID)
java.util.Set<DataFile>
fetchNewDataFiles(Table table, java.lang.String fileSetID)
java.util.Set<java.lang.String>
fetchSetIDs(Table table)
static FileRewriteCoordinator
get()
void
stageRewrite(Table table, java.lang.String fileSetID, java.util.Set<DataFile> newDataFiles)
Called to persist the output of a rewrite action for a specific group.
-
-
-
Method Detail
-
get
public static FileRewriteCoordinator get()
-
stageRewrite
public void stageRewrite(Table table, java.lang.String fileSetID, java.util.Set<DataFile> newDataFiles)
Called to persist the output of a rewrite action for a specific group. Since the write is done via a Spark Datasource, we have to propagate the result through this side-effect call.- Parameters:
table
- table where the rewrite is occurringfileSetID
- the id used to identify the source set of files being rewrittennewDataFiles
- the new files which have been written
-
fetchNewDataFiles
public java.util.Set<DataFile> fetchNewDataFiles(Table table, java.lang.String fileSetID)
-
clearRewrite
public void clearRewrite(Table table, java.lang.String fileSetID)
-
fetchSetIDs
public java.util.Set<java.lang.String> fetchSetIDs(Table table)
-
-