org.apache.iceberg.flink.source.reader

Class ColumnStatsWatermarkExtractor

java.lang.Object

org.apache.iceberg.flink.source.reader.ColumnStatsWatermarkExtractor

All Implemented Interfaces:

java.io.Serializable, SplitWatermarkExtractor

@Internal
public class ColumnStatsWatermarkExtractor
extends java.lang.Object
implements SplitWatermarkExtractor, java.io.Serializable

SplitWatermarkExtractor implementation which uses an Iceberg timestamp column statistics to get the watermarks for the IcebergSourceSplit. This watermark is emitted by the WatermarkExtractorRecordEmitter along with the actual records.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ColumnStatsWatermarkExtractor(Schema schema, java.lang.String eventTimeFieldName, java.util.concurrent.TimeUnit timeUnit) Creates the extractor.

Method Summary

Modifier and Type	Method and Description
long	extractWatermark(IcebergSourceSplit split) Get the watermark for a split using column statistics.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ColumnStatsWatermarkExtractor

public ColumnStatsWatermarkExtractor(Schema schema,
                                     java.lang.String eventTimeFieldName,
                                     java.util.concurrent.TimeUnit timeUnit)

Creates the extractor.

Parameters:

schema - The schema of the Table

eventTimeFieldName - The column which should be used as an event time

timeUnit - Used for converting the long value to epoch milliseconds

Method Detail

extractWatermark

public long extractWatermark(IcebergSourceSplit split)

Get the watermark for a split using column statistics.

Specified by:

extractWatermark in interface SplitWatermarkExtractor

Parameters:

split - The split

Returns:

The watermark

Throws:

java.lang.IllegalArgumentException - if there is no statistics for the column