Package org.apache.iceberg.actions
Class BinPackRewritePositionDeletePlanner
java.lang.Object
org.apache.iceberg.actions.SizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>
org.apache.iceberg.actions.BinPackRewritePositionDeletePlanner
- All Implemented Interfaces:
FileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,
PositionDeletesScanTask, DeleteFile, RewritePositionDeletesGroup>
public class BinPackRewritePositionDeletePlanner
extends SizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask,DeleteFile,RewritePositionDeletesGroup>
Groups specified delete files in the
Table
into RewritePositionDeletesGroup
s.
These will be grouped by partitions based on their size using fix sized bins. Extends the SizeBasedFileRewritePlanner
with RewritePositionDeleteFiles.REWRITE_JOB_ORDER
handling.-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.iceberg.actions.SizeBasedFileRewritePlanner
SizeBasedFileRewritePlanner.RewriteExecutionContext
-
Field Summary
Fields inherited from class org.apache.iceberg.actions.SizeBasedFileRewritePlanner
MAX_FILE_GROUP_SIZE_BYTES, MAX_FILE_GROUP_SIZE_BYTES_DEFAULT, MAX_FILE_SIZE_BYTES, MAX_FILE_SIZE_DEFAULT_RATIO, MIN_FILE_SIZE_BYTES, MIN_FILE_SIZE_DEFAULT_RATIO, MIN_INPUT_FILES, MIN_INPUT_FILES_DEFAULT, REWRITE_ALL, REWRITE_ALL_DEFAULT, TARGET_FILE_SIZE_BYTES
-
Constructor Summary
ConstructorsConstructorDescriptionBinPackRewritePositionDeletePlanner
(Table table, Expression filter, boolean caseSensitive) Creates the planner for the given table. -
Method Summary
Modifier and TypeMethodDescriptionprotected long
Expected target file size before configuration.protected Iterable<List<PositionDeletesScanTask>>
filterFileGroups
(List<List<PositionDeletesScanTask>> groups) Additional filter for groups.protected Iterable<PositionDeletesScanTask>
Additional filter for tasks before grouping.void
Initializes this planner using provided options.FileRewritePlan<RewritePositionDeleteFiles.FileGroupInfo,
PositionDeletesScanTask, DeleteFile, RewritePositionDeletesGroup> plan()
Generates the plan for rewrite.Returns a set of supported options for this planner.Methods inherited from class org.apache.iceberg.actions.SizeBasedFileRewritePlanner
enoughContent, enoughInputFiles, expectedOutputFiles, inputSize, inputSplitSize, outputSpecId, outsideDesiredFileSizeRange, planFileGroups, table, tooMuchContent, writeMaxFileSize
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.iceberg.actions.FileRewritePlanner
description
-
Constructor Details
-
BinPackRewritePositionDeletePlanner
-
BinPackRewritePositionDeletePlanner
Creates the planner for the given table.- Parameters:
table
- to plan forfilter
- used to remove files from the plancaseSensitive
- property used for scanning
-
-
Method Details
-
validOptions
Description copied from interface:FileRewritePlanner
Returns a set of supported options for this planner. Only options specified in this list will be accepted at runtime. Any other options will be rejected.- Specified by:
validOptions
in interfaceFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,
PositionDeletesScanTask, DeleteFile, RewritePositionDeletesGroup> - Overrides:
validOptions
in classSizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,
PositionDeletesScanTask, DeleteFile, RewritePositionDeletesGroup>
-
init
Description copied from interface:FileRewritePlanner
Initializes this planner using provided options.- Specified by:
init
in interfaceFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,
PositionDeletesScanTask, DeleteFile, RewritePositionDeletesGroup> - Overrides:
init
in classSizeBasedFileRewritePlanner<RewritePositionDeleteFiles.FileGroupInfo,
PositionDeletesScanTask, DeleteFile, RewritePositionDeletesGroup> - Parameters:
options
- options to initialize this planner
-
plan
public FileRewritePlan<RewritePositionDeleteFiles.FileGroupInfo,PositionDeletesScanTask, plan()DeleteFile, RewritePositionDeletesGroup> Description copied from interface:FileRewritePlanner
Generates the plan for rewrite.- Returns:
- the generated plan which could be executed during the compaction
-
filterFiles
Description copied from class:SizeBasedFileRewritePlanner
Additional filter for tasks before grouping. -
filterFileGroups
protected Iterable<List<PositionDeletesScanTask>> filterFileGroups(List<List<PositionDeletesScanTask>> groups) Description copied from class:SizeBasedFileRewritePlanner
Additional filter for groups. -
defaultTargetFileSize
protected long defaultTargetFileSize()Description copied from class:SizeBasedFileRewritePlanner
Expected target file size before configuration.
-