Package org.apache.iceberg.actions

Class BinPackRewritePositionDeletePlanner

java.lang.Object

org.apache.iceberg.actions.SizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>

org.apache.iceberg.actions.BinPackRewritePositionDeletePlanner

All Implemented Interfaces:

FileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>

public class BinPackRewritePositionDeletePlanner
extends SizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>

Groups specified delete files in the Table into RewritePositionDeletesGroups. These will be grouped by partitions based on their size using fix sized bins. Extends the SizeBasedFileRewritePlanner with RewritePositionDeleteFiles.REWRITE_JOB_ORDER handling.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.iceberg.actions.SizeBasedFileRewritePlanner

SizeBasedFileRewritePlanner.RewriteExecutionContext

Field Summary

Fields inherited from class org.apache.iceberg.actions.SizeBasedFileRewritePlanner

MAX_FILE_GROUP_SIZE_BYTES, MAX_FILE_GROUP_SIZE_BYTES_DEFAULT, MAX_FILE_SIZE_BYTES, MAX_FILE_SIZE_DEFAULT_RATIO, MIN_FILE_SIZE_BYTES, MIN_FILE_SIZE_DEFAULT_RATIO, MIN_INPUT_FILES, MIN_INPUT_FILES_DEFAULT, REWRITE_ALL, REWRITE_ALL_DEFAULT, TARGET_FILE_SIZE_BYTES

Constructor Summary

Constructors

Constructor	Description
BinPackRewritePositionDeletePlanner(Table table)
BinPackRewritePositionDeletePlanner(Table table, Expression filter, boolean caseSensitive)	Creates the planner for the given table.

Method Summary

Modifier and Type	Method	Description
protected long	defaultTargetFileSize()	Expected target file size before configuration.
protected Iterable<List<PositionDeletesScanTask>>	filterFileGroups(List<List<PositionDeletesScanTask>> groups)	Additional filter for groups.
protected Iterable<PositionDeletesScanTask>	filterFiles(Iterable<PositionDeletesScanTask> tasks)	Additional filter for tasks before grouping.
void	init(Map<String,String> options)	Initializes this planner using provided options.
FileRewritePlan<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>	plan()	Generates the plan for rewrite.
Set<String>	validOptions()	Returns a set of supported options for this planner.

Methods inherited from class org.apache.iceberg.actions.SizeBasedFileRewritePlanner

enoughContent, enoughInputFiles, expectedOutputFiles, inputSize, inputSplitSize, outputSpecId, outsideDesiredFileSizeRange, planFileGroups, table, tooMuchContent, writeMaxFileSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.actions.FileRewritePlanner

description

Constructor Details

BinPackRewritePositionDeletePlanner

public BinPackRewritePositionDeletePlanner(Table table)

BinPackRewritePositionDeletePlanner

public BinPackRewritePositionDeletePlanner(Table table,
 Expression filter,
 boolean caseSensitive)

Creates the planner for the given table.

Parameters:

table - to plan for

filter - used to remove files from the plan

caseSensitive - property used for scanning

Method Details

validOptions

public Set<String> validOptions()

Description copied from interface: FileRewritePlanner

Returns a set of supported options for this planner. Only options specified in this list will be accepted at runtime. Any other options will be rejected.

Specified by:

validOptions in interface FileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>

Overrides:

validOptions in class SizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>

init

public void init(Map<String,String> options)

Description copied from interface: FileRewritePlanner

Initializes this planner using provided options.

Specified by:

init in interface FileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>

Overrides:

init in class SizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>

Parameters:

options - options to initialize this planner

plan

public FileRewritePlan<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup> plan()

Description copied from interface: FileRewritePlanner

Generates the plan for rewrite.

Returns:

the generated plan which could be executed during the compaction

filterFiles

protected Iterable<PositionDeletesScanTask> filterFiles(Iterable<PositionDeletesScanTask> tasks)

Description copied from class: SizeBasedFileRewritePlanner

Additional filter for tasks before grouping.

Specified by:

filterFiles in class SizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>

filterFileGroups

protected Iterable<List<PositionDeletesScanTask>> filterFileGroups(List<List<PositionDeletesScanTask>> groups)

Description copied from class: SizeBasedFileRewritePlanner

Additional filter for groups.

Specified by:

filterFileGroups in class SizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>

defaultTargetFileSize

protected long defaultTargetFileSize()

Description copied from class: SizeBasedFileRewritePlanner

Expected target file size before configuration.

Specified by:

defaultTargetFileSize in class SizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>