org.apache.iceberg.spark.actions

Class Spark3BinPackStrategy

java.lang.Object

org.apache.iceberg.actions.BinPackStrategy

org.apache.iceberg.spark.actions.Spark3BinPackStrategy

All Implemented Interfaces:

java.io.Serializable, RewriteStrategy

public class Spark3BinPackStrategy
extends BinPackStrategy

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.iceberg.actions.BinPackStrategy

DELETE_FILE_THRESHOLD, DELETE_FILE_THRESHOLD_DEFAULT, MAX_FILE_SIZE_BYTES, MAX_FILE_SIZE_DEFAULT_RATIO, MIN_FILE_SIZE_BYTES, MIN_FILE_SIZE_DEFAULT_RATIO, MIN_INPUT_FILES, MIN_INPUT_FILES_DEFAULT

Constructor Summary

Constructors

Constructor and Description
Spark3BinPackStrategy(Table table, org.apache.spark.sql.SparkSession spark)

Method Summary

Modifier and Type	Method and Description
java.util.Set<DataFile>	rewriteFiles(java.util.List<FileScanTask> filesToRewrite) Method which will rewrite files based on this particular RewriteStrategy's algorithm.
Table	table() Returns the table being modified by this rewrite strategy

Methods inherited from class org.apache.iceberg.actions.BinPackStrategy

inputFileSize, maxGroupSize, name, numOutputFiles, options, planFileGroups, selectFilesToRewrite, splitSize, targetFileSize, validOptions, writeMaxFileSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Spark3BinPackStrategy

public Spark3BinPackStrategy(Table table,
                             org.apache.spark.sql.SparkSession spark)

Method Detail

table

public Table table()

Description copied from interface: RewriteStrategy

Returns the table being modified by this rewrite strategy

rewriteFiles

public java.util.Set<DataFile> rewriteFiles(java.util.List<FileScanTask> filesToRewrite)

Description copied from interface: RewriteStrategy

Method which will rewrite files based on this particular RewriteStrategy's algorithm. This will most likely be Action framework specific (Spark/Presto/Flink ....).

Parameters:

filesToRewrite - a group of files to be rewritten together

Returns:

a set of newly written files