Package org.apache.iceberg.spark.actions
Class SparkZOrderStrategy
- java.lang.Object
-
- org.apache.iceberg.actions.BinPackStrategy
-
- org.apache.iceberg.actions.SortStrategy
-
- org.apache.iceberg.spark.actions.SparkSortStrategy
-
- org.apache.iceberg.spark.actions.SparkZOrderStrategy
-
- All Implemented Interfaces:
java.io.Serializable
,RewriteStrategy
public class SparkZOrderStrategy extends SparkSortStrategy
- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from class org.apache.iceberg.spark.actions.SparkSortStrategy
COMPRESSION_FACTOR
-
Fields inherited from class org.apache.iceberg.actions.BinPackStrategy
DELETE_FILE_THRESHOLD, DELETE_FILE_THRESHOLD_DEFAULT, MAX_FILE_SIZE_BYTES, MAX_FILE_SIZE_DEFAULT_RATIO, MIN_FILE_SIZE_BYTES, MIN_FILE_SIZE_DEFAULT_RATIO, MIN_INPUT_FILES, MIN_INPUT_FILES_DEFAULT, REWRITE_ALL, REWRITE_ALL_DEFAULT
-
-
Constructor Summary
Constructors Constructor Description SparkZOrderStrategy(Table table, org.apache.spark.sql.SparkSession spark, java.util.List<java.lang.String> zOrderColNames)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.lang.String
name()
Returns the name of this rewrite strategyRewriteStrategy
options(java.util.Map<java.lang.String,java.lang.String> options)
Sets options to be used with this strategyjava.util.Set<DataFile>
rewriteFiles(java.util.List<FileScanTask> filesToRewrite)
Method which will rewrite files based on this particular RewriteStrategy's algorithm.protected SortOrder
sortOrder()
protected void
validateOptions()
java.util.Set<java.lang.String>
validOptions()
Returns a set of options which this rewrite strategy can use.-
Methods inherited from class org.apache.iceberg.spark.actions.SparkSortStrategy
manager, rewriteCoordinator, sizeEstimateMultiple, sortPlan, spark, table, tableCache
-
Methods inherited from class org.apache.iceberg.actions.SortStrategy
sortOrder
-
Methods inherited from class org.apache.iceberg.actions.BinPackStrategy
inputFileSize, numOutputFiles, planFileGroups, selectFilesToRewrite, splitSize, targetFileSize, writeMaxFileSize
-
-
-
-
Constructor Detail
-
SparkZOrderStrategy
public SparkZOrderStrategy(Table table, org.apache.spark.sql.SparkSession spark, java.util.List<java.lang.String> zOrderColNames)
-
-
Method Detail
-
validOptions
public java.util.Set<java.lang.String> validOptions()
Description copied from interface:RewriteStrategy
Returns a set of options which this rewrite strategy can use. This is an allowed-list and any options not specified here will be rejected at runtime.- Specified by:
validOptions
in interfaceRewriteStrategy
- Overrides:
validOptions
in classSparkSortStrategy
-
options
public RewriteStrategy options(java.util.Map<java.lang.String,java.lang.String> options)
Description copied from interface:RewriteStrategy
Sets options to be used with this strategy- Specified by:
options
in interfaceRewriteStrategy
- Overrides:
options
in classSparkSortStrategy
-
name
public java.lang.String name()
Description copied from interface:RewriteStrategy
Returns the name of this rewrite strategy- Specified by:
name
in interfaceRewriteStrategy
- Overrides:
name
in classSortStrategy
-
validateOptions
protected void validateOptions()
- Overrides:
validateOptions
in classSortStrategy
-
rewriteFiles
public java.util.Set<DataFile> rewriteFiles(java.util.List<FileScanTask> filesToRewrite)
Description copied from interface:RewriteStrategy
Method which will rewrite files based on this particular RewriteStrategy's algorithm. This will most likely be Action framework specific (Spark/Presto/Flink ....).- Specified by:
rewriteFiles
in interfaceRewriteStrategy
- Overrides:
rewriteFiles
in classSparkSortStrategy
- Parameters:
filesToRewrite
- a group of files to be rewritten together- Returns:
- a set of newly written files
-
sortOrder
protected SortOrder sortOrder()
- Overrides:
sortOrder
in classSortStrategy
-
-