Package org.apache.iceberg.spark

Class ComputeUpdateIterator

java.lang.Object

org.apache.iceberg.spark.ChangelogIterator

org.apache.iceberg.spark.ComputeUpdateIterator

All Implemented Interfaces:

Iterator<org.apache.spark.sql.Row>

public class ComputeUpdateIterator
extends ChangelogIterator

An iterator that finds delete/insert rows which represent an update, and converts them into update records from changelog tables within a single Spark task. It assumes that rows are sorted by identifier columns and change type.

For example, these two rows

• (id=1, data='a', op='DELETE')
• (id=1, data='b', op='INSERT')

will be marked as update-rows:

• (id=1, data='a', op='UPDATE_BEFORE')
• (id=1, data='b', op='UPDATE_AFTER')

Field Summary

Fields inherited from class org.apache.iceberg.spark.ChangelogIterator

DELETE, INSERT, UPDATE_AFTER, UPDATE_BEFORE

Method Summary

Modifier and Type	Method	Description
boolean	hasNext()
org.apache.spark.sql.Row	next()

Methods inherited from class org.apache.iceberg.spark.ChangelogIterator

changeType, changeTypeIndex, computeUpdates, generateIndicesToIdentifySameRow, isDifferentValue, isSameRecord, removeCarryovers, removeNetCarryovers, rowIterator, rowType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Method Details

hasNext

public boolean hasNext()

next

public org.apache.spark.sql.Row next()