org.apache.iceberg.flink.sink

Class FlinkSink.Builder

java.lang.Object

org.apache.iceberg.flink.sink.FlinkSink.Builder

Enclosing class:

FlinkSink

public static class FlinkSink.Builder
extends java.lang.Object

Method Summary

Modifier and Type	Method and Description
org.apache.flink.streaming.api.datastream.DataStreamSink<org.apache.flink.table.data.RowData>	build()
FlinkSink.Builder	distributionMode(DistributionMode mode) Configure the write DistributionMode that the flink sink will use.
FlinkSink.Builder	equalityFieldColumns(java.util.List<java.lang.String> columns) Configuring the equality field columns for iceberg table that accept CDC or UPSERT events.
FlinkSink.Builder	overwrite(boolean newOverwrite)
FlinkSink.Builder	table(Table newTable) This iceberg Table instance is used for initializing IcebergStreamWriter which will write all the records into DataFiles and emit them to downstream operator.
FlinkSink.Builder	tableLoader(TableLoader newTableLoader) The table loader is used for loading tables in IcebergFilesCommitter lazily, we need this loader because Table is not serializable and could not just use the loaded table from Builder#table in the remote task manager.
FlinkSink.Builder	tableSchema(org.apache.flink.table.api.TableSchema newTableSchema)
FlinkSink.Builder	writeParallelism(int newWriteParallelism) Configuring the write parallel number for iceberg stream writer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

table

public FlinkSink.Builder table(Table newTable)

This iceberg Table instance is used for initializing IcebergStreamWriter which will write all the records into DataFiles and emit them to downstream operator. Providing a table would avoid so many table loading from each separate task.

Parameters:

newTable - the loaded iceberg table instance.

Returns:

FlinkSink.Builder to connect the iceberg table.

tableLoader

public FlinkSink.Builder tableLoader(TableLoader newTableLoader)

The table loader is used for loading tables in IcebergFilesCommitter lazily, we need this loader because Table is not serializable and could not just use the loaded table from Builder#table in the remote task manager.

Parameters:

newTableLoader - to load iceberg table inside tasks.

Returns:

FlinkSink.Builder to connect the iceberg table.

tableSchema

public FlinkSink.Builder tableSchema(org.apache.flink.table.api.TableSchema newTableSchema)

overwrite

public FlinkSink.Builder overwrite(boolean newOverwrite)

distributionMode

public FlinkSink.Builder distributionMode(DistributionMode mode)

Configure the write DistributionMode that the flink sink will use. Currently, flink support DistributionMode.NONE and DistributionMode.HASH.

Parameters:

mode - to specify the write distribution mode.

Returns:

FlinkSink.Builder to connect the iceberg table.

writeParallelism

public FlinkSink.Builder writeParallelism(int newWriteParallelism)

Configuring the write parallel number for iceberg stream writer.

Parameters:

newWriteParallelism - the number of parallel iceberg stream writer.

Returns:

FlinkSink.Builder to connect the iceberg table.

equalityFieldColumns

public FlinkSink.Builder equalityFieldColumns(java.util.List<java.lang.String> columns)

Configuring the equality field columns for iceberg table that accept CDC or UPSERT events.

Parameters:

columns - defines the iceberg table's key.

Returns:

FlinkSink.Builder to connect the iceberg table.

build

public org.apache.flink.streaming.api.datastream.DataStreamSink<org.apache.flink.table.data.RowData> build()