Package org.apache.iceberg.actions

Class RewriteDataFilesCommitManager

java.lang.Object

org.apache.iceberg.actions.RewriteDataFilesCommitManager

public class RewriteDataFilesCommitManager
extends Object

Functionality used by RewriteDataFile Actions from different platforms to handle commits.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	RewriteDataFilesCommitManager.CommitService

Constructor Summary

Constructors

Constructor	Description
RewriteDataFilesCommitManager(Table table)
RewriteDataFilesCommitManager(Table table, long startingSnapshotId)
RewriteDataFilesCommitManager(Table table, long startingSnapshotId, boolean useStartingSequenceNumber)
RewriteDataFilesCommitManager(Table table, long startingSnapshotId, boolean useStartingSequenceNumber, Map<String,String> snapshotProperties)

Method Summary

Modifier and Type	Method	Description
void	abortFileGroup(RewriteFileGroup fileGroup)	Clean up a specified file set by removing any files created for that operation, should not throw any exceptions
void	commitFileGroups(Set<RewriteFileGroup> fileGroups)	Perform a commit operation on the table adding and removing files as required for this set of file groups
void	commitOrClean(Set<RewriteFileGroup> rewriteGroups)
RewriteDataFilesCommitManager.CommitService	service(int rewritesPerCommit)	An async service which allows for committing multiple file groups as their rewrites complete.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

RewriteDataFilesCommitManager

public RewriteDataFilesCommitManager(Table table)

RewriteDataFilesCommitManager

public RewriteDataFilesCommitManager(Table table,
 long startingSnapshotId)

RewriteDataFilesCommitManager

public RewriteDataFilesCommitManager(Table table,
 long startingSnapshotId,
 boolean useStartingSequenceNumber)

RewriteDataFilesCommitManager

public RewriteDataFilesCommitManager(Table table,
 long startingSnapshotId,
 boolean useStartingSequenceNumber,
 Map<String,String> snapshotProperties)

Method Details

commitFileGroups

public void commitFileGroups(Set<RewriteFileGroup> fileGroups)

Perform a commit operation on the table adding and removing files as required for this set of file groups

Parameters:

fileGroups - fileSets to commit

abortFileGroup

public void abortFileGroup(RewriteFileGroup fileGroup)

Clean up a specified file set by removing any files created for that operation, should not throw any exceptions

Parameters:

fileGroup - group of files which has already been rewritten

commitOrClean

public void commitOrClean(Set<RewriteFileGroup> rewriteGroups)

service

public RewriteDataFilesCommitManager.CommitService service(int rewritesPerCommit)

An async service which allows for committing multiple file groups as their rewrites complete. The service also allows for partial-progress since commits can fail. Once the service has been closed no new file groups should not be offered.

Parameters:

rewritesPerCommit - number of file groups to include in a commit

Returns:

the service for handling commits