public class BaseReplacePartitions extends java.lang.Object implements ReplacePartitions
Modifier and Type | Method and Description |
---|---|
protected void |
add(DataFile file)
Add a data file to the new snapshot.
|
protected void |
add(DeleteFile file)
Add a delete file to the new snapshot.
|
protected void |
add(ManifestFile manifest)
Add all files in a manifest to the new snapshot.
|
protected java.util.List<DataFile> |
addedFiles() |
ReplacePartitions |
addFile(DataFile file)
Add a
DataFile to the table. |
Snapshot |
apply()
Apply the pending changes and return the uncommitted changes for validation.
|
java.util.List<ManifestFile> |
apply(TableMetadata base)
Apply the update's changes to the base table metadata and return the new manifest list.
|
protected void |
cleanAll() |
protected void |
cleanUncommitted(java.util.Set<ManifestFile> committed)
Clean up any uncommitted manifests that were created.
|
void |
commit()
Apply the pending changes and commit.
|
protected TableMetadata |
current() |
protected void |
delete(java.lang.CharSequence path)
Add a specific data path to be deleted in the new snapshot.
|
protected void |
delete(DataFile file)
Add a specific data file to be deleted in the new snapshot.
|
protected void |
delete(DeleteFile file)
Add a specific delete file to be deleted in the new snapshot.
|
protected void |
deleteByRowFilter(Expression expr)
Add a filter to match files to delete.
|
protected void |
deleteFile(java.lang.String path) |
ThisT |
deleteWith(java.util.function.Consumer<java.lang.String> deleteCallback)
Set a callback to delete files instead of the table's default.
|
protected void |
dropPartition(int specId,
StructLike partition)
Add a partition tuple to drop from the table during the delete phase.
|
protected void |
failAnyDelete() |
protected void |
failMissingDeletePaths() |
protected OutputFile |
manifestListPath() |
protected ManifestReader<DeleteFile> |
newDeleteManifestReader(ManifestFile manifest) |
protected ManifestWriter<DeleteFile> |
newDeleteManifestWriter(PartitionSpec spec) |
protected OutputFile |
newManifestOutput() |
protected ManifestReader<DataFile> |
newManifestReader(ManifestFile manifest) |
protected ManifestWriter<DataFile> |
newManifestWriter(PartitionSpec spec) |
protected java.lang.String |
operation()
A string that describes the action that produced the new snapshot.
|
protected TableMetadata |
refresh() |
protected Expression |
rowFilter() |
protected ReplacePartitions |
self() |
ThisT |
set(java.lang.String property,
java.lang.String value)
Set a summary property in the snapshot produced by this update.
|
protected long |
snapshotId() |
ThisT |
stageOnly()
Called to stage a snapshot in table metadata, but not update the current snapshot id.
|
protected java.util.Map<java.lang.String,java.lang.String> |
summary() |
java.lang.Object |
updateEvent()
Generates update event to notify about metadata changes
|
protected void |
validate(TableMetadata currentMetadata)
Validate the current metadata.
|
protected void |
validateAddedDataFiles(TableMetadata base,
java.lang.Long startingSnapshotId,
Expression conflictDetectionFilter,
boolean caseSensitive)
Validates that no files matching a filter have been added to the table since a starting snapshot.
|
ReplacePartitions |
validateAppendOnly()
Validate that no partitions will be replaced and the operation is append-only.
|
protected void |
validateDataFilesExist(TableMetadata base,
java.lang.Long startingSnapshotId,
java.util.Set<java.lang.CharSequence> requiredDataFiles,
boolean skipDeletes) |
protected PartitionSpec |
writeSpec() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
deleteWith, set, stageOnly
apply, commit, updateEvent
protected ReplacePartitions self()
protected java.lang.String operation()
public ReplacePartitions addFile(DataFile file)
ReplacePartitions
DataFile
to the table.addFile
in interface ReplacePartitions
file
- a data filepublic ReplacePartitions validateAppendOnly()
ReplacePartitions
validateAppendOnly
in interface ReplacePartitions
public java.util.List<ManifestFile> apply(TableMetadata base)
base
- the base table metadata to apply changes topublic ThisT set(java.lang.String property, java.lang.String value)
SnapshotUpdate
property
- a String property namevalue
- a String property valueprotected PartitionSpec writeSpec()
protected Expression rowFilter()
protected java.util.List<DataFile> addedFiles()
protected void failAnyDelete()
protected void failMissingDeletePaths()
protected void deleteByRowFilter(Expression expr)
expr
- an expression to match rows.protected void dropPartition(int specId, StructLike partition)
protected void delete(DataFile file)
protected void delete(DeleteFile file)
protected void delete(java.lang.CharSequence path)
protected void add(DataFile file)
protected void add(DeleteFile file)
protected void add(ManifestFile manifest)
protected void validateAddedDataFiles(TableMetadata base, java.lang.Long startingSnapshotId, Expression conflictDetectionFilter, boolean caseSensitive)
base
- table metadata to validatestartingSnapshotId
- id of the snapshot current at the start of the operationconflictDetectionFilter
- an expression used to find new conflicting data filescaseSensitive
- whether expression evaluation should be case sensitiveprotected void validateDataFilesExist(TableMetadata base, java.lang.Long startingSnapshotId, java.util.Set<java.lang.CharSequence> requiredDataFiles, boolean skipDeletes)
protected java.util.Map<java.lang.String,java.lang.String> summary()
public java.lang.Object updateEvent()
PendingUpdate
protected void cleanUncommitted(java.util.Set<ManifestFile> committed)
Manifests may not be committed if apply is called more because a commit conflict has occurred. Implementations may keep around manifests because the same changes will be made by both apply calls. This method instructs the implementation to clean up those manifests and passes the paths of the manifests that were actually committed.
committed
- a set of manifest paths that were actually committedpublic ThisT stageOnly()
SnapshotUpdate
stageOnly
in interface SnapshotUpdate<ThisT>
public ThisT deleteWith(java.util.function.Consumer<java.lang.String> deleteCallback)
SnapshotUpdate
deleteWith
in interface SnapshotUpdate<ThisT>
deleteCallback
- a String consumer used to delete locations.protected void validate(TableMetadata currentMetadata)
Child operations can override this to add custom validation.
currentMetadata
- current table metadata to validatepublic Snapshot apply()
PendingUpdate
This does not result in a permanent update.
apply
in interface PendingUpdate<Snapshot>
PendingUpdate.commit()
protected TableMetadata current()
protected TableMetadata refresh()
public void commit()
PendingUpdate
Changes are committed by calling the underlying table's commit method.
Once the commit is successful, the updated table will be refreshed.
commit
in interface PendingUpdate<Snapshot>
protected void cleanAll()
protected void deleteFile(java.lang.String path)
protected OutputFile manifestListPath()
protected OutputFile newManifestOutput()
protected ManifestWriter<DataFile> newManifestWriter(PartitionSpec spec)
protected ManifestWriter<DeleteFile> newDeleteManifestWriter(PartitionSpec spec)
protected ManifestReader<DataFile> newManifestReader(ManifestFile manifest)
protected ManifestReader<DeleteFile> newDeleteManifestReader(ManifestFile manifest)
protected long snapshotId()