Package org.apache.iceberg.flink.source
Class StreamingMonitorFunction
- java.lang.Object
-
- org.apache.flink.api.common.functions.AbstractRichFunction
-
- org.apache.flink.streaming.api.functions.source.RichSourceFunction<FlinkInputSplit>
-
- org.apache.iceberg.flink.source.StreamingMonitorFunction
-
- All Implemented Interfaces:
java.io.Serializable
,org.apache.flink.api.common.functions.Function
,org.apache.flink.api.common.functions.RichFunction
,org.apache.flink.streaming.api.checkpoint.CheckpointedFunction
,org.apache.flink.streaming.api.functions.source.SourceFunction<FlinkInputSplit>
public class StreamingMonitorFunction extends org.apache.flink.streaming.api.functions.source.RichSourceFunction<FlinkInputSplit> implements org.apache.flink.streaming.api.checkpoint.CheckpointedFunction
This is the single (non-parallel) monitoring task which takes aFlinkInputFormat
, it is responsible for:- Monitoring snapshots of the Iceberg table.
- Creating the
splits
corresponding to the incremental files - Assigning them to downstream tasks for further processing.
The splits to be read are forwarded to the downstream
StreamingReaderOperator
which can have parallelism greater than one.- See Also:
- Serialized Form
-
-
Constructor Summary
Constructors Constructor Description StreamingMonitorFunction(TableLoader tableLoader, org.apache.iceberg.flink.source.ScanContext scanContext)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
cancel()
void
close()
void
initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
void
run(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<FlinkInputSplit> ctx)
void
snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)
-
-
-
Constructor Detail
-
StreamingMonitorFunction
public StreamingMonitorFunction(TableLoader tableLoader, org.apache.iceberg.flink.source.ScanContext scanContext)
-
-
Method Detail
-
initializeState
public void initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context) throws java.lang.Exception
- Specified by:
initializeState
in interfaceorg.apache.flink.streaming.api.checkpoint.CheckpointedFunction
- Throws:
java.lang.Exception
-
snapshotState
public void snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context) throws java.lang.Exception
- Specified by:
snapshotState
in interfaceorg.apache.flink.streaming.api.checkpoint.CheckpointedFunction
- Throws:
java.lang.Exception
-
run
public void run(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<FlinkInputSplit> ctx) throws java.lang.Exception
- Specified by:
run
in interfaceorg.apache.flink.streaming.api.functions.source.SourceFunction<FlinkInputSplit>
- Throws:
java.lang.Exception
-
cancel
public void cancel()
- Specified by:
cancel
in interfaceorg.apache.flink.streaming.api.functions.source.SourceFunction<FlinkInputSplit>
-
close
public void close()
- Specified by:
close
in interfaceorg.apache.flink.api.common.functions.RichFunction
- Overrides:
close
in classorg.apache.flink.api.common.functions.AbstractRichFunction
-
-