org.apache.iceberg.spark.actions

Class SparkBinPackStrategy

java.lang.Object

org.apache.iceberg.actions.BinPackStrategy

org.apache.iceberg.spark.actions.SparkBinPackStrategy

All Implemented Interfaces:

java.io.Serializable, RewriteStrategy

public class SparkBinPackStrategy
extends BinPackStrategy

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.iceberg.actions.BinPackStrategy

DELETE_FILE_THRESHOLD, DELETE_FILE_THRESHOLD_DEFAULT, MAX_FILE_SIZE_BYTES, MAX_FILE_SIZE_DEFAULT_RATIO, MIN_FILE_SIZE_BYTES, MIN_FILE_SIZE_DEFAULT_RATIO, MIN_INPUT_FILES, MIN_INPUT_FILES_DEFAULT, REWRITE_ALL, REWRITE_ALL_DEFAULT

Constructor Summary

Constructors

Constructor and Description
SparkBinPackStrategy(Table table, org.apache.spark.sql.SparkSession spark)

Method Summary

Modifier and Type	Method and Description
java.util.Set<DataFile>	rewriteFiles(java.util.List<FileScanTask> filesToRewrite) Method which will rewrite files based on this particular RewriteStrategy's algorithm.
Table	table() Returns the table being modified by this rewrite strategy

Methods inherited from class org.apache.iceberg.actions.BinPackStrategy

inputFileSize, name, numOutputFiles, options, planFileGroups, selectFilesToRewrite, splitSize, targetFileSize, validOptions, writeMaxFileSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkBinPackStrategy

public SparkBinPackStrategy(Table table,
                            org.apache.spark.sql.SparkSession spark)

Method Detail

table

public Table table()

Description copied from interface: RewriteStrategy

Returns the table being modified by this rewrite strategy

rewriteFiles

public java.util.Set<DataFile> rewriteFiles(java.util.List<FileScanTask> filesToRewrite)

Description copied from interface: RewriteStrategy

Method which will rewrite files based on this particular RewriteStrategy's algorithm. This will most likely be Action framework specific (Spark/Presto/Flink ....).

Parameters:

filesToRewrite - a group of files to be rewritten together

Returns:

a set of newly written files