org.apache.iceberg.spark.actions

Class SparkZOrderStrategy

java.lang.Object

org.apache.iceberg.actions.BinPackStrategy

org.apache.iceberg.actions.SortStrategy

org.apache.iceberg.spark.actions.SparkSortStrategy

org.apache.iceberg.spark.actions.SparkZOrderStrategy

All Implemented Interfaces:

java.io.Serializable, RewriteStrategy

public class SparkZOrderStrategy
extends SparkSortStrategy

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.iceberg.spark.actions.SparkSortStrategy

COMPRESSION_FACTOR

Fields inherited from class org.apache.iceberg.actions.BinPackStrategy

DELETE_FILE_THRESHOLD, DELETE_FILE_THRESHOLD_DEFAULT, MAX_FILE_SIZE_BYTES, MAX_FILE_SIZE_DEFAULT_RATIO, MIN_FILE_SIZE_BYTES, MIN_FILE_SIZE_DEFAULT_RATIO, MIN_INPUT_FILES, MIN_INPUT_FILES_DEFAULT, REWRITE_ALL, REWRITE_ALL_DEFAULT

Constructor Summary

Constructors

Constructor and Description
SparkZOrderStrategy(Table table, org.apache.spark.sql.SparkSession spark, java.util.List<java.lang.String> zOrderColNames)

Method Summary

Modifier and Type	Method and Description
java.lang.String	name() Returns the name of this rewrite strategy
RewriteStrategy	options(java.util.Map<java.lang.String,java.lang.String> options) Sets options to be used with this strategy
java.util.Set<DataFile>	rewriteFiles(java.util.List<FileScanTask> filesToRewrite) Method which will rewrite files based on this particular RewriteStrategy's algorithm.
protected SortOrder	sortOrder()
protected void	validateOptions()
java.util.Set<java.lang.String>	validOptions() Returns a set of options which this rewrite strategy can use.

Methods inherited from class org.apache.iceberg.spark.actions.SparkSortStrategy

manager, rewriteCoordinator, sizeEstimateMultiple, sortPlan, spark, table, tableCache

Methods inherited from class org.apache.iceberg.actions.SortStrategy

sortOrder

Methods inherited from class org.apache.iceberg.actions.BinPackStrategy

inputFileSize, numOutputFiles, planFileGroups, selectFilesToRewrite, splitSize, targetFileSize, writeMaxFileSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkZOrderStrategy

public SparkZOrderStrategy(Table table,
                           org.apache.spark.sql.SparkSession spark,
                           java.util.List<java.lang.String> zOrderColNames)

Method Detail

validOptions

public java.util.Set<java.lang.String> validOptions()

Description copied from interface: RewriteStrategy

Returns a set of options which this rewrite strategy can use. This is an allowed-list and any options not specified here will be rejected at runtime.

Specified by:

validOptions in interface RewriteStrategy

Overrides:

validOptions in class SparkSortStrategy

options

public RewriteStrategy options(java.util.Map<java.lang.String,java.lang.String> options)

Description copied from interface: RewriteStrategy

Sets options to be used with this strategy

Specified by:

options in interface RewriteStrategy

Overrides:

options in class SparkSortStrategy

name

public java.lang.String name()

Description copied from interface: RewriteStrategy

Returns the name of this rewrite strategy

Specified by:

name in interface RewriteStrategy

Overrides:

name in class SortStrategy

validateOptions

protected void validateOptions()

Overrides:

validateOptions in class SortStrategy

rewriteFiles

public java.util.Set<DataFile> rewriteFiles(java.util.List<FileScanTask> filesToRewrite)

Description copied from interface: RewriteStrategy

Method which will rewrite files based on this particular RewriteStrategy's algorithm. This will most likely be Action framework specific (Spark/Presto/Flink ....).

Specified by:

rewriteFiles in interface RewriteStrategy

Overrides:

rewriteFiles in class SparkSortStrategy

Parameters:

filesToRewrite - a group of files to be rewritten together

Returns:

a set of newly written files

sortOrder

protected SortOrder sortOrder()

Overrides:

sortOrder in class SortStrategy