public class SnapshotManager extends java.lang.Object implements ManageSnapshots
Modifier and Type | Method and Description |
---|---|
protected void |
add(DataFile file)
Add a data file to the new snapshot.
|
protected void |
add(DeleteFile file)
Add a delete file to the new snapshot.
|
protected void |
add(ManifestFile manifest)
Add all files in a manifest to the new snapshot.
|
protected java.util.List<DataFile> |
addedFiles() |
Snapshot |
apply()
Apply the pending changes and return the uncommitted changes for validation.
|
java.util.List<ManifestFile> |
apply(TableMetadata base)
Apply the update's changes to the base table metadata and return the new manifest list.
|
ManageSnapshots |
cherrypick(long snapshotId)
Apply supported changes in given snapshot and create a new snapshot which will be set as the
current snapshot on commit.
|
protected void |
cleanAll() |
protected void |
cleanUncommitted(java.util.Set<ManifestFile> committed)
Clean up any uncommitted manifests that were created.
|
void |
commit()
Apply the pending changes and commit.
|
protected TableMetadata |
current() |
protected void |
delete(java.lang.CharSequence path)
Add a specific data path to be deleted in the new snapshot.
|
protected void |
delete(DataFile file)
Add a specific data file to be deleted in the new snapshot.
|
protected void |
delete(DeleteFile file)
Add a specific delete file to be deleted in the new snapshot.
|
protected void |
deleteByRowFilter(Expression expr)
Add a filter to match files to delete.
|
protected void |
deleteFile(java.lang.String path) |
ThisT |
deleteWith(java.util.function.Consumer<java.lang.String> deleteCallback)
Set a callback to delete files instead of the table's default.
|
protected void |
dropPartition(int specId,
StructLike partition)
Add a partition tuple to drop from the table during the delete phase.
|
protected void |
failAnyDelete() |
protected void |
failMissingDeletePaths() |
protected OutputFile |
manifestListPath() |
protected ManifestReader<DeleteFile> |
newDeleteManifestReader(ManifestFile manifest) |
protected ManifestWriter<DeleteFile> |
newDeleteManifestWriter(PartitionSpec spec) |
protected OutputFile |
newManifestOutput() |
protected ManifestReader<DataFile> |
newManifestReader(ManifestFile manifest) |
protected ManifestWriter<DataFile> |
newManifestWriter(PartitionSpec spec) |
protected java.lang.String |
operation()
A string that describes the action that produced the new snapshot.
|
protected TableMetadata |
refresh() |
ManageSnapshots |
rollbackTo(long snapshotId)
Rollback table's state to a specific
Snapshot identified by id. |
ManageSnapshots |
rollbackToTime(long timestampMillis)
Roll this table's data back to the last
Snapshot before the given timestamp. |
protected Expression |
rowFilter() |
protected ManageSnapshots |
self() |
ThisT |
set(java.lang.String property,
java.lang.String value)
Set a summary property in the snapshot produced by this update.
|
ManageSnapshots |
setCurrentSnapshot(long snapshotId)
Roll this table's data back to a specific
Snapshot identified by id. |
protected long |
snapshotId() |
ThisT |
stageOnly()
Called to stage a snapshot in table metadata, but not update the current snapshot id.
|
protected java.util.Map<java.lang.String,java.lang.String> |
summary() |
java.lang.Object |
updateEvent()
Generates update event to notify about metadata changes
|
protected void |
validate(TableMetadata base)
Validate the current metadata.
|
protected void |
validateAddedDataFiles(TableMetadata base,
java.lang.Long startingSnapshotId,
Expression conflictDetectionFilter,
boolean caseSensitive)
Validates that no files matching a filter have been added to the table since a starting snapshot.
|
protected void |
validateDataFilesExist(TableMetadata base,
java.lang.Long startingSnapshotId,
java.util.Set<java.lang.CharSequence> requiredDataFiles,
boolean skipDeletes) |
protected PartitionSpec |
writeSpec() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
commit
protected ManageSnapshots self()
protected java.lang.String operation()
public ManageSnapshots cherrypick(long snapshotId)
ManageSnapshots
cherrypick
in interface ManageSnapshots
snapshotId
- a snapshotId whose changes to applypublic ManageSnapshots setCurrentSnapshot(long snapshotId)
ManageSnapshots
Snapshot
identified by id.setCurrentSnapshot
in interface ManageSnapshots
snapshotId
- long id of the snapshot to roll back table data topublic ManageSnapshots rollbackToTime(long timestampMillis)
ManageSnapshots
Snapshot
before the given timestamp.rollbackToTime
in interface ManageSnapshots
timestampMillis
- a long timestamp, as returned by System.currentTimeMillis()
public ManageSnapshots rollbackTo(long snapshotId)
ManageSnapshots
Snapshot
identified by id.rollbackTo
in interface ManageSnapshots
snapshotId
- long id of snapshot id to roll back table to. Must be an ancestor of the current snapshotpublic java.lang.Object updateEvent()
PendingUpdate
updateEvent
in interface PendingUpdate<Snapshot>
protected void validate(TableMetadata base)
Child operations can override this to add custom validation.
base
- current table metadata to validatepublic Snapshot apply()
PendingUpdate
This does not result in a permanent update.
apply
in interface PendingUpdate<Snapshot>
PendingUpdate.commit()
public ThisT set(java.lang.String property, java.lang.String value)
SnapshotUpdate
property
- a String property namevalue
- a String property valueprotected PartitionSpec writeSpec()
protected Expression rowFilter()
protected java.util.List<DataFile> addedFiles()
protected void failAnyDelete()
protected void failMissingDeletePaths()
protected void deleteByRowFilter(Expression expr)
expr
- an expression to match rows.protected void dropPartition(int specId, StructLike partition)
protected void delete(DataFile file)
protected void delete(DeleteFile file)
protected void delete(java.lang.CharSequence path)
protected void add(DataFile file)
protected void add(DeleteFile file)
protected void add(ManifestFile manifest)
protected void validateAddedDataFiles(TableMetadata base, java.lang.Long startingSnapshotId, Expression conflictDetectionFilter, boolean caseSensitive)
base
- table metadata to validatestartingSnapshotId
- id of the snapshot current at the start of the operationconflictDetectionFilter
- an expression used to find new conflicting data filescaseSensitive
- whether expression evaluation should be case sensitiveprotected void validateDataFilesExist(TableMetadata base, java.lang.Long startingSnapshotId, java.util.Set<java.lang.CharSequence> requiredDataFiles, boolean skipDeletes)
protected java.util.Map<java.lang.String,java.lang.String> summary()
public java.util.List<ManifestFile> apply(TableMetadata base)
base
- the base table metadata to apply changes toprotected void cleanUncommitted(java.util.Set<ManifestFile> committed)
Manifests may not be committed if apply is called more because a commit conflict has occurred. Implementations may keep around manifests because the same changes will be made by both apply calls. This method instructs the implementation to clean up those manifests and passes the paths of the manifests that were actually committed.
committed
- a set of manifest paths that were actually committedpublic ThisT stageOnly()
SnapshotUpdate
stageOnly
in interface SnapshotUpdate<ThisT>
public ThisT deleteWith(java.util.function.Consumer<java.lang.String> deleteCallback)
SnapshotUpdate
deleteWith
in interface SnapshotUpdate<ThisT>
deleteCallback
- a String consumer used to delete locations.protected TableMetadata current()
protected TableMetadata refresh()
public void commit()
PendingUpdate
Changes are committed by calling the underlying table's commit method.
Once the commit is successful, the updated table will be refreshed.
commit
in interface PendingUpdate<Snapshot>
protected void cleanAll()
protected void deleteFile(java.lang.String path)
protected OutputFile manifestListPath()
protected OutputFile newManifestOutput()
protected ManifestWriter<DataFile> newManifestWriter(PartitionSpec spec)
protected ManifestWriter<DeleteFile> newDeleteManifestWriter(PartitionSpec spec)
protected ManifestReader<DataFile> newManifestReader(ManifestFile manifest)
protected ManifestReader<DeleteFile> newDeleteManifestReader(ManifestFile manifest)
protected long snapshotId()