Class AllDeleteFilesTable.AllDeleteFilesTableScan
- java.lang.Object
-
- org.apache.iceberg.AllDeleteFilesTable.AllDeleteFilesTableScan
-
- All Implemented Interfaces:
Scan<TableScan,FileScanTask,CombinedScanTask>,TableScan
- Enclosing class:
- AllDeleteFilesTable
public static class AllDeleteFilesTable.AllDeleteFilesTableScan extends java.lang.Object
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description TableScanappendsAfter(long fromSnapshotId)Create a newTableScanto read appended data fromfromSnapshotIdexclusive to the current snapshot inclusive.TableScanappendsBetween(long fromSnapshotId, long toSnapshotId)Create a newTableScanto read appended data fromfromSnapshotIdexclusive totoSnapshotIdinclusive.TableScanasOfTime(long timestampMillis)Create a newTableScanfrom this scan's configuration that will use the most recent snapshot as of the given time in milliseconds.ThisTcaseSensitive(boolean caseSensitive)Create a new scan from this that, if data columns where selected viaScan.select(java.util.Collection), controls whether the match to the schema will be done with case sensitivity.protected booleancolStats()protected org.apache.iceberg.TableScanContextcontext()protected CloseableIterable<FileScanTask>doPlanFiles()Expressionfilter()Returns this scan's filterExpression.ThisTfilter(Expression expr)Create a new scan from the results of this filtered by theExpression.ThisTignoreResiduals()Create a new scan from this that applies data filtering to files but not to rows in those files.ThisTincludeColumnStats()Create a new scan from this that loads the column stats with each data file.booleanisCaseSensitive()Returns whether this scan should apply column name case sensitiveness as perScan.caseSensitive(boolean).protected CloseableIterable<ManifestFile>manifests()Returns an iterable of manifest files to explore for this all files metadata table scanprotected TableScannewRefinedScan(TableOperations ops, Table table, Schema schema, org.apache.iceberg.TableScanContext context)ThisToption(java.lang.String property, java.lang.String value)Create a new scan from this scan's configuration that will override theTable's behavior based on the incoming pair.protected java.util.Map<java.lang.String,java.lang.String>options()protected java.util.concurrent.ExecutorServiceplanExecutor()CloseableIterable<FileScanTask>planFiles()Plan tasks for this scan where each task reads a single file.CloseableIterable<CombinedScanTask>planTasks()Plan balanced task groups for this scan by splitting large and combining small tasks.ThisTplanWith(java.util.concurrent.ExecutorService executorService)Create a new scan to use a particular executor to plan.ThisTproject(Schema projectedSchema)Create a new scan from this with the schema as its projection.protected CloseableIterable<ManifestFile>reachableManifests(org.apache.iceberg.relocated.com.google.common.base.Function<Snapshot,java.lang.Iterable<ManifestFile>> toManifests)Schemaschema()Returns this scan's projectionSchema.ThisTselect(java.util.Collection<java.lang.String> columns)Create a new scan from this that will read the given data columns.protected booleanshouldIgnoreResiduals()Snapshotsnapshot()Returns theSnapshotthat will be used by this scan.protected java.lang.LongsnapshotId()intsplitLookback()Returns the split lookback for this scan.longsplitOpenFileCost()Returns the split open file cost for this scan.Tabletable()Returns theTablefrom which this scan loads data.protected TableOperationstableOps()protected SchematableSchema()protected MetadataTableTypetableType()Type of scan being performed, such asMetadataTableType.ALL_DATA_FILESwhen scanning a table'sAllDataFilesTable.longtargetSplitSize()Returns the target split size for this scan.java.lang.StringtoString()TableScanuseSnapshot(long scanSnapshotId)Create a newTableScanfrom this scan's configuration that will use the given snapshot by ID.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.apache.iceberg.Scan
caseSensitive, filter, ignoreResiduals, includeColumnStats, option, planWith, project, schema, select, splitLookback, splitOpenFileCost
-
-
-
-
Method Detail
-
newRefinedScan
protected TableScan newRefinedScan(TableOperations ops, Table table, Schema schema, org.apache.iceberg.TableScanContext context)
-
manifests
protected CloseableIterable<ManifestFile> manifests()
Returns an iterable of manifest files to explore for this all files metadata table scan
-
doPlanFiles
protected CloseableIterable<FileScanTask> doPlanFiles()
-
useSnapshot
public TableScan useSnapshot(long scanSnapshotId)
Description copied from interface:TableScanCreate a newTableScanfrom this scan's configuration that will use the given snapshot by ID.- Specified by:
useSnapshotin interfaceTableScan- Parameters:
scanSnapshotId- a snapshot ID- Returns:
- a new scan based on this with the given snapshot ID
-
asOfTime
public TableScan asOfTime(long timestampMillis)
Description copied from interface:TableScanCreate a newTableScanfrom this scan's configuration that will use the most recent snapshot as of the given time in milliseconds.
-
planFiles
public CloseableIterable<FileScanTask> planFiles()
Description copied from interface:ScanPlan tasks for this scan where each task reads a single file.Use
Scan.planTasks()for planning balanced tasks where each task will read either a single file, a part of a file, or multiple files.- Specified by:
planFilesin interfaceScan<TableScan,FileScanTask,CombinedScanTask>- Returns:
- an Iterable of tasks scanning entire files required by this scan
-
reachableManifests
protected CloseableIterable<ManifestFile> reachableManifests(org.apache.iceberg.relocated.com.google.common.base.Function<Snapshot,java.lang.Iterable<ManifestFile>> toManifests)
-
tableType
protected MetadataTableType tableType()
Type of scan being performed, such asMetadataTableType.ALL_DATA_FILESwhen scanning a table'sAllDataFilesTable.Used for logging and error messages.
-
appendsBetween
public TableScan appendsBetween(long fromSnapshotId, long toSnapshotId)
Description copied from interface:TableScanCreate a newTableScanto read appended data fromfromSnapshotIdexclusive totoSnapshotIdinclusive.- Specified by:
appendsBetweenin interfaceTableScan- Parameters:
fromSnapshotId- the last snapshot id read by the user, exclusivetoSnapshotId- read append data up to this snapshot id- Returns:
- a table scan which can read append data from
fromSnapshotIdexclusive and up totoSnapshotIdinclusive
-
appendsAfter
public TableScan appendsAfter(long fromSnapshotId)
Description copied from interface:TableScanCreate a newTableScanto read appended data fromfromSnapshotIdexclusive to the current snapshot inclusive.- Specified by:
appendsAfterin interfaceTableScan- Parameters:
fromSnapshotId- - the last snapshot id read by the user, exclusive- Returns:
- a table scan which can read append data from
fromSnapshotIdexclusive and up to current snapshot inclusive
-
targetSplitSize
public long targetSplitSize()
Description copied from interface:ScanReturns the target split size for this scan.- Specified by:
targetSplitSizein interfaceScan<TableScan,FileScanTask,CombinedScanTask>
-
snapshotId
protected java.lang.Long snapshotId()
-
colStats
protected boolean colStats()
-
shouldIgnoreResiduals
protected boolean shouldIgnoreResiduals()
-
planExecutor
protected java.util.concurrent.ExecutorService planExecutor()
-
options
protected java.util.Map<java.lang.String,java.lang.String> options()
-
table
public Table table()
Description copied from interface:TableScanReturns theTablefrom which this scan loads data.
-
filter
public Expression filter()
Description copied from interface:TableScanReturns this scan's filterExpression.
-
planTasks
public CloseableIterable<CombinedScanTask> planTasks()
Description copied from interface:ScanPlan balanced task groups for this scan by splitting large and combining small tasks.Task groups created by this method may read partial input files, multiple input files or both.
- Specified by:
planTasksin interfaceScan<TableScan,FileScanTask,CombinedScanTask>- Returns:
- an Iterable of balanced task groups required by this scan
-
snapshot
public Snapshot snapshot()
Description copied from interface:TableScanReturns theSnapshotthat will be used by this scan.If the snapshot was not configured using
TableScan.asOfTime(long)orTableScan.useSnapshot(long), the current table snapshot will be used.
-
isCaseSensitive
public boolean isCaseSensitive()
Description copied from interface:TableScanReturns whether this scan should apply column name case sensitiveness as perScan.caseSensitive(boolean).- Specified by:
isCaseSensitivein interfaceTableScan- Returns:
- true if case sensitive, false otherwise.
-
toString
public java.lang.String toString()
- Overrides:
toStringin classjava.lang.Object
-
tableOps
protected TableOperations tableOps()
-
tableSchema
protected Schema tableSchema()
-
context
protected org.apache.iceberg.TableScanContext context()
-
option
public ThisT option(java.lang.String property, java.lang.String value)Description copied from interface:ScanCreate a new scan from this scan's configuration that will override theTable's behavior based on the incoming pair. Unknown properties will be ignored.- Specified by:
optionin interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>- Parameters:
property- name of the table property to be overriddenvalue- value to override with- Returns:
- a new scan based on this with overridden behavior
-
project
public ThisT project(Schema projectedSchema)
Description copied from interface:ScanCreate a new scan from this with the schema as its projection.- Specified by:
projectin interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>- Parameters:
projectedSchema- a projection schema- Returns:
- a new scan based on this with the given projection
-
caseSensitive
public ThisT caseSensitive(boolean caseSensitive)
Description copied from interface:ScanCreate a new scan from this that, if data columns where selected viaScan.select(java.util.Collection), controls whether the match to the schema will be done with case sensitivity. Default is true.- Specified by:
caseSensitivein interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>- Returns:
- a new scan based on this with case sensitivity as stated
-
includeColumnStats
public ThisT includeColumnStats()
Description copied from interface:ScanCreate a new scan from this that loads the column stats with each data file.Column stats include: value count, null value count, lower bounds, and upper bounds.
- Specified by:
includeColumnStatsin interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>- Returns:
- a new scan based on this that loads column stats.
-
select
public ThisT select(java.util.Collection<java.lang.String> columns)
Description copied from interface:ScanCreate a new scan from this that will read the given data columns. This produces an expected schema that includes all fields that are either selected or used by this scan's filter expression.- Specified by:
selectin interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>- Parameters:
columns- column names from the table's schema- Returns:
- a new scan based on this with the given projection columns
-
filter
public ThisT filter(Expression expr)
Description copied from interface:ScanCreate a new scan from the results of this filtered by theExpression.- Specified by:
filterin interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>- Parameters:
expr- a filter expression- Returns:
- a new scan based on this with results filtered by the expression
-
ignoreResiduals
public ThisT ignoreResiduals()
Description copied from interface:ScanCreate a new scan from this that applies data filtering to files but not to rows in those files.- Specified by:
ignoreResidualsin interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>- Returns:
- a new scan based on this that does not filter rows in files.
-
planWith
public ThisT planWith(java.util.concurrent.ExecutorService executorService)
Description copied from interface:ScanCreate a new scan to use a particular executor to plan. The default worker pool will be used by default.- Specified by:
planWithin interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>- Parameters:
executorService- the provided executor- Returns:
- a table scan that uses the provided executor to access manifests
-
schema
public Schema schema()
Description copied from interface:ScanReturns this scan's projectionSchema.If the projection schema was set directly using
Scan.project(Schema), returns that schema.If the projection schema was set by calling
Scan.select(Collection), returns a projection schema that includes the selected data fields and any fields used in the filter expression.- Specified by:
schemain interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>- Returns:
- this scan's projection schema
-
splitLookback
public int splitLookback()
Description copied from interface:ScanReturns the split lookback for this scan.- Specified by:
splitLookbackin interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>
-
splitOpenFileCost
public long splitOpenFileCost()
Description copied from interface:ScanReturns the split open file cost for this scan.- Specified by:
splitOpenFileCostin interfaceScan<ThisT,T extends ScanTask,G extends ScanTaskGroup<T>>
-
-