Package org.apache.iceberg.spark.actions
Class Spark3BinPackStrategy
- java.lang.Object
-
- org.apache.iceberg.actions.BinPackStrategy
-
- org.apache.iceberg.spark.actions.Spark3BinPackStrategy
-
- All Implemented Interfaces:
java.io.Serializable
,RewriteStrategy
public class Spark3BinPackStrategy extends BinPackStrategy
- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from class org.apache.iceberg.actions.BinPackStrategy
DELETE_FILE_THRESHOLD, DELETE_FILE_THRESHOLD_DEFAULT, MAX_FILE_SIZE_BYTES, MAX_FILE_SIZE_DEFAULT_RATIO, MIN_FILE_SIZE_BYTES, MIN_FILE_SIZE_DEFAULT_RATIO, MIN_INPUT_FILES, MIN_INPUT_FILES_DEFAULT
-
-
Constructor Summary
Constructors Constructor Description Spark3BinPackStrategy(Table table, org.apache.spark.sql.SparkSession spark)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.util.Set<DataFile>
rewriteFiles(java.util.List<FileScanTask> filesToRewrite)
Method which will rewrite files based on this particular RewriteStrategy's algorithm.Table
table()
Returns the table being modified by this rewrite strategy-
Methods inherited from class org.apache.iceberg.actions.BinPackStrategy
inputFileSize, maxGroupSize, name, numOutputFiles, options, planFileGroups, selectFilesToRewrite, splitSize, targetFileSize, validOptions, writeMaxFileSize
-
-
-
-
Constructor Detail
-
Spark3BinPackStrategy
public Spark3BinPackStrategy(Table table, org.apache.spark.sql.SparkSession spark)
-
-
Method Detail
-
table
public Table table()
Description copied from interface:RewriteStrategy
Returns the table being modified by this rewrite strategy
-
rewriteFiles
public java.util.Set<DataFile> rewriteFiles(java.util.List<FileScanTask> filesToRewrite)
Description copied from interface:RewriteStrategy
Method which will rewrite files based on this particular RewriteStrategy's algorithm. This will most likely be Action framework specific (Spark/Presto/Flink ....).- Parameters:
filesToRewrite
- a group of files to be rewritten together- Returns:
- a set of newly written files
-
-