Package org.apache.iceberg.actions

Interface FileRewritePlanner<I,T extends ContentScanTask<F>,F extends ContentFile<F>,G extends RewriteGroupBase<I,T,F>>

Type Parameters:
I - the Java type of the plan info like RewriteDataFiles.FileGroupInfo or RewritePositionDeleteFiles.FileGroupInfo
T - the Java type of the input scan tasks (input)
F - the Java type of the content files (input and output)
G - the Java type of the rewrite file group like RewriteFileGroup or RewritePositionDeletesGroup
All Known Implementing Classes:
BinPackRewriteFilePlanner, BinPackRewritePositionDeletePlanner, SizeBasedFileRewritePlanner
public interface FileRewritePlanner<I,T extends ContentScanTask<F>,F extends ContentFile<F>,G extends RewriteGroupBase<I,T,F>>
A class for planning a rewrite operation.

A Rewrite Operation is performed by several classes working in conjunction.

  • FileRewritePlanner - (this class) which scans the files in the table and determines which files should be rewritten and how they should be grouped. The grouping is based on partitioning and the planning could create multiple groups within a partition. This produces a FileRewritePlan.
  • FileRewritePlan - This immutable output of the planner contains a set of groups like RewriteFileGroup or RewritePositionDeletesGroup, each containing a set of files that are meant to be compacted together by the FileRewriteRunner.
  • FileRewriteRunner - The runner is the engine specific implementation that can take a FileRewritePlan and perform a rewrite on each of the groups within. This is the actual implementation of the rewrite which generates new files.

The lifecycle for the planner looks like the following:

  • Method Summary

    Modifier and Type
    Method
    Description
    default String
    description()
    Returns a description for this planner.
    void
    init(Map<String,String> options)
    Initializes this planner using provided options.
    FileRewritePlan<I,T,F,G>
    plan()
    Generates the plan for rewrite.
    Set<String>
    validOptions()
    Returns a set of supported options for this planner.

  • Method Details

    • description

      default String description()
      Returns a description for this planner.

    • validOptions

      Set<String> validOptions()
      Returns a set of supported options for this planner. Only options specified in this list will be accepted at runtime. Any other options will be rejected.

    • init

      void init(Map<String,String> options)
      Initializes this planner using provided options.
      Parameters:
      options - options to initialize this planner

    • plan

      FileRewritePlan<I,T,F,G> plan()
      Generates the plan for rewrite.
      Returns:
      the generated plan which could be executed during the compaction