Package org.apache.iceberg.flink.sink.dynamic

Class EvolveSchemaVisitor

java.lang.Object

org.apache.iceberg.schema.SchemaWithPartnerVisitor<Integer,Boolean>

org.apache.iceberg.flink.sink.dynamic.EvolveSchemaVisitor

public class EvolveSchemaVisitor
extends SchemaWithPartnerVisitor<Integer,Boolean>

Visitor class that accumulates the set of changes needed to evolve an existing schema into the target schema. Changes are applied to an UpdateSchema operation.

We support:

• Adding new columns
• Widening the type of existing columsn
• Reordering columns
• Dropping columns (when dropUnusedColumns is enabled)

We don't support:

• Renaming columns

By default, any columns present in the table but absent from the input schema are marked as optional to prevent issues caused by late or out-of-order data. If dropUnusedColumns is enabled, these columns are removed instead to ensure a strict one-to-one schema alignment.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.iceberg.schema.SchemaWithPartnerVisitor

SchemaWithPartnerVisitor.PartnerAccessors<P>

Method Summary

Modifier and Type	Method	Description
Boolean	field(Types.NestedField field, Integer partnerId, Boolean isFieldMissing)
Boolean	list(Types.ListType list, Integer partnerId, Boolean isElementMissing)
Boolean	map(Types.MapType map, Integer partnerId, Boolean isKeyMissing, Boolean isValueMissing)
Boolean	primitive(Type.PrimitiveType primitive, Integer partnerId)
Boolean	struct(Types.StructType struct, Integer partnerId, List<Boolean> existingFields)
static void	visit(TableIdentifier identifier, UpdateSchema api, Schema existingSchema, Schema targetSchema, boolean caseSensitive, boolean dropUnusedColumns)	Adds changes needed to produce the target schema to an UpdateSchema operation.

Methods inherited from class org.apache.iceberg.schema.SchemaWithPartnerVisitor

afterField, afterListElement, afterMapKey, afterMapValue, beforeField, beforeListElement, beforeMapKey, beforeMapValue, schema, variant, visit, visit

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

visit

public static void visit(TableIdentifier identifier,
 UpdateSchema api,
 Schema existingSchema,
 Schema targetSchema,
 boolean caseSensitive,
 boolean dropUnusedColumns)

Adds changes needed to produce the target schema to an UpdateSchema operation.

Changes are accumulated to evolve the existingSchema into a targetSchema.

Parameters:

api - an UpdateSchema for adding changes

existingSchema - an existing schema

targetSchema - a new schema to compare with the existing

caseSensitive - whether field name matching should be case-sensitive

dropUnusedColumns - whether to drop columns not present in target schema

struct

public Boolean struct(Types.StructType struct,
 Integer partnerId,
 List<Boolean> existingFields)

Overrides:

struct in class SchemaWithPartnerVisitor<Integer,Boolean>

field

public Boolean field(Types.NestedField field,
 Integer partnerId,
 Boolean isFieldMissing)

Overrides:

field in class SchemaWithPartnerVisitor<Integer,Boolean>

list

public Boolean list(Types.ListType list,
 Integer partnerId,
 Boolean isElementMissing)

Overrides:

list in class SchemaWithPartnerVisitor<Integer,Boolean>

map

public Boolean map(Types.MapType map,
 Integer partnerId,
 Boolean isKeyMissing,
 Boolean isValueMissing)

Overrides:

map in class SchemaWithPartnerVisitor<Integer,Boolean>

primitive

public Boolean primitive(Type.PrimitiveType primitive,
 Integer partnerId)

Overrides:

primitive in class SchemaWithPartnerVisitor<Integer,Boolean>