Interface UpdateSchema
-
- All Superinterfaces:
PendingUpdate<Schema>
public interface UpdateSchema extends PendingUpdate<Schema>
API for schema evolution.When committing, these changes will be applied to the current table metadata. Commit conflicts will not be resolved and will result in a
CommitFailedException
.
-
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description default UpdateSchema
addColumn(java.lang.String parent, java.lang.String name, Type type)
Add a new column to a nested struct.UpdateSchema
addColumn(java.lang.String parent, java.lang.String name, Type type, java.lang.String doc)
Add a new column to a nested struct.default UpdateSchema
addColumn(java.lang.String name, Type type)
Add a new top-level column.UpdateSchema
addColumn(java.lang.String name, Type type, java.lang.String doc)
Add a new top-level column.default UpdateSchema
addRequiredColumn(java.lang.String parent, java.lang.String name, Type type)
Add a new required top-level column.UpdateSchema
addRequiredColumn(java.lang.String parent, java.lang.String name, Type type, java.lang.String doc)
Add a new required top-level column.default UpdateSchema
addRequiredColumn(java.lang.String name, Type type)
Add a new required top-level column.UpdateSchema
addRequiredColumn(java.lang.String name, Type type, java.lang.String doc)
Add a new required top-level column.UpdateSchema
allowIncompatibleChanges()
Allow incompatible changes to the schema.default UpdateSchema
caseSensitive(boolean caseSensitive)
Determines if the case of schema needs to be considered when comparing column namesUpdateSchema
deleteColumn(java.lang.String name)
Delete a column in the schema.UpdateSchema
makeColumnOptional(java.lang.String name)
Update a column to optional.UpdateSchema
moveAfter(java.lang.String name, java.lang.String afterName)
Move a column from its current position to directly after a reference column.UpdateSchema
moveBefore(java.lang.String name, java.lang.String beforeName)
Move a column from its current position to directly before a reference column.UpdateSchema
moveFirst(java.lang.String name)
Move a column from its current position to the start of the schema or its parent struct.UpdateSchema
renameColumn(java.lang.String name, java.lang.String newName)
Rename a column in the schema.UpdateSchema
requireColumn(java.lang.String name)
Update a column to required.default UpdateSchema
setIdentifierFields(java.lang.String... names)
Set the identifier fields given some field names.UpdateSchema
setIdentifierFields(java.util.Collection<java.lang.String> names)
Set the identifier fields given a set of field names.UpdateSchema
unionByNameWith(Schema newSchema)
Applies all field additions and updates from the provided new schema to the existing schema so to create a union schema.UpdateSchema
updateColumn(java.lang.String name, Type.PrimitiveType newType)
Update a column in the schema to a new primitive type.default UpdateSchema
updateColumn(java.lang.String name, Type.PrimitiveType newType, java.lang.String newDoc)
Update a column in the schema to a new primitive type.UpdateSchema
updateColumnDoc(java.lang.String name, java.lang.String newDoc)
Update a column in the schema to a new primitive type.-
Methods inherited from interface org.apache.iceberg.PendingUpdate
apply, commit, updateEvent
-
-
-
-
Method Detail
-
allowIncompatibleChanges
UpdateSchema allowIncompatibleChanges()
Allow incompatible changes to the schema.Incompatible changes can cause failures when attempting to read older data files. For example, adding a required column and attempting to read data files without that column will cause a failure. However, if there are no data files that are not compatible with the change, it can be allowed.
This option allows incompatible changes to be made to a schema. This should be used when the caller has validated that the change will not break. For example, if a column is added as optional but always populated and data older than the column addition has been deleted from the table, this can be used with
requireColumn(String)
to mark the column required.- Returns:
- this for method chaining
-
addColumn
default UpdateSchema addColumn(java.lang.String name, Type type)
Add a new top-level column.Because "." may be interpreted as a column path separator or may be used in field names, it is not allowed in names passed to this method. To add to nested structures or to add fields with names that contain ".", use
addColumn(String, String, Type)
.If type is a nested type, its field IDs are reassigned when added to the existing schema.
- Parameters:
name
- name for the new columntype
- type for the new column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name contains "."
-
addColumn
UpdateSchema addColumn(java.lang.String name, Type type, java.lang.String doc)
Add a new top-level column.Because "." may be interpreted as a column path separator or may be used in field names, it is not allowed in names passed to this method. To add to nested structures or to add fields with names that contain ".", use
addColumn(String, String, Type)
.If type is a nested type, its field IDs are reassigned when added to the existing schema.
- Parameters:
name
- name for the new columntype
- type for the new columndoc
- documentation string for the new column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name contains "."
-
addColumn
default UpdateSchema addColumn(java.lang.String parent, java.lang.String name, Type type)
Add a new column to a nested struct.The parent name is used to find the parent using
Schema.findField(String)
. If the parent name is null, the new column will be added to the root as a top-level column. If parent identifies a struct, a new column is added to that struct. If it identifies a list, the column is added to the list element struct, and if it identifies a map, the new column is added to the map's value struct.The given name is used to name the new column and names containing "." are not handled differently.
If type is a nested type, its field IDs are reassigned when added to the existing schema.
- Parameters:
parent
- name of the parent struct to the column will be added toname
- name for the new columntype
- type for the new column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If parent doesn't identify a struct
-
addColumn
UpdateSchema addColumn(java.lang.String parent, java.lang.String name, Type type, java.lang.String doc)
Add a new column to a nested struct.The parent name is used to find the parent using
Schema.findField(String)
. If the parent name is null, the new column will be added to the root as a top-level column. If parent identifies a struct, a new column is added to that struct. If it identifies a list, the column is added to the list element struct, and if it identifies a map, the new column is added to the map's value struct.The given name is used to name the new column and names containing "." are not handled differently.
If type is a nested type, its field IDs are reassigned when added to the existing schema.
- Parameters:
parent
- name of the parent struct to the column will be added toname
- name for the new columntype
- type for the new columndoc
- documentation string for the new column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If parent doesn't identify a struct
-
addRequiredColumn
default UpdateSchema addRequiredColumn(java.lang.String name, Type type)
Add a new required top-level column.This is an incompatible change that can break reading older data. This method will result in an exception unless
allowIncompatibleChanges()
has been called.Because "." may be interpreted as a column path separator or may be used in field names, it is not allowed in names passed to this method. To add to nested structures or to add fields with names that contain ".", use
addRequiredColumn(String, String, Type)
.If type is a nested type, its field IDs are reassigned when added to the existing schema.
- Parameters:
name
- name for the new columntype
- type for the new column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name contains "."
-
addRequiredColumn
UpdateSchema addRequiredColumn(java.lang.String name, Type type, java.lang.String doc)
Add a new required top-level column.This is an incompatible change that can break reading older data. This method will result in an exception unless
allowIncompatibleChanges()
has been called.Because "." may be interpreted as a column path separator or may be used in field names, it is not allowed in names passed to this method. To add to nested structures or to add fields with names that contain ".", use
addRequiredColumn(String, String, Type)
.If type is a nested type, its field IDs are reassigned when added to the existing schema.
- Parameters:
name
- name for the new columntype
- type for the new columndoc
- documentation string for the new column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name contains "."
-
addRequiredColumn
default UpdateSchema addRequiredColumn(java.lang.String parent, java.lang.String name, Type type)
Add a new required top-level column.This is an incompatible change that can break reading older data. This method will result in an exception unless
allowIncompatibleChanges()
has been called.The parent name is used to find the parent using
Schema.findField(String)
. If the parent name is null, the new column will be added to the root as a top-level column. If parent identifies a struct, a new column is added to that struct. If it identifies a list, the column is added to the list element struct, and if it identifies a map, the new column is added to the map's value struct.The given name is used to name the new column and names containing "." are not handled differently.
If type is a nested type, its field IDs are reassigned when added to the existing schema.
- Parameters:
parent
- name of the parent struct to the column will be added toname
- name for the new columntype
- type for the new column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If parent doesn't identify a struct
-
addRequiredColumn
UpdateSchema addRequiredColumn(java.lang.String parent, java.lang.String name, Type type, java.lang.String doc)
Add a new required top-level column.This is an incompatible change that can break reading older data. This method will result in an exception unless
allowIncompatibleChanges()
has been called.The parent name is used to find the parent using
Schema.findField(String)
. If the parent name is null, the new column will be added to the root as a top-level column. If parent identifies a struct, a new column is added to that struct. If it identifies a list, the column is added to the list element struct, and if it identifies a map, the new column is added to the map's value struct.The given name is used to name the new column and names containing "." are not handled differently.
If type is a nested type, its field IDs are reassigned when added to the existing schema.
- Parameters:
parent
- name of the parent struct to the column will be added toname
- name for the new columntype
- type for the new columndoc
- documentation string for the new column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If parent doesn't identify a struct
-
renameColumn
UpdateSchema renameColumn(java.lang.String name, java.lang.String newName)
Rename a column in the schema.The name is used to find the column to rename using
Schema.findField(String)
.The new name may contain "." and such names are not parsed or handled differently.
Columns may be updated and renamed in the same schema update.
- Parameters:
name
- name of the column to renamenewName
- replacement name for the column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name doesn't identify a column in the schema or if this change conflicts with other additions, renames, or updates.
-
updateColumn
UpdateSchema updateColumn(java.lang.String name, Type.PrimitiveType newType)
Update a column in the schema to a new primitive type.The name is used to find the column to update using
Schema.findField(String)
.Only updates that widen types are allowed.
Columns may be updated and renamed in the same schema update.
- Parameters:
name
- name of the column to renamenewType
- replacement type for the column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name doesn't identify a column in the schema or if this change introduces a type incompatibility or if it conflicts with other additions, renames, or updates.
-
updateColumn
default UpdateSchema updateColumn(java.lang.String name, Type.PrimitiveType newType, java.lang.String newDoc)
Update a column in the schema to a new primitive type.The name is used to find the column to update using
Schema.findField(String)
.Only updates that widen types are allowed.
Columns may be updated and renamed in the same schema update.
- Parameters:
name
- name of the column to renamenewType
- replacement type for the columnnewDoc
- replacement documentation string for the column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name doesn't identify a column in the schema or if this change introduces a type incompatibility or if it conflicts with other additions, renames, or updates.
-
updateColumnDoc
UpdateSchema updateColumnDoc(java.lang.String name, java.lang.String newDoc)
Update a column in the schema to a new primitive type.The name is used to find the column to update using
Schema.findField(String)
.Columns may be updated and renamed in the same schema update.
- Parameters:
name
- name of the column to renamenewDoc
- replacement documentation string for the column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name doesn't identify a column in the schema or if this change introduces a type incompatibility or if it conflicts with other additions, renames, or updates.
-
makeColumnOptional
UpdateSchema makeColumnOptional(java.lang.String name)
Update a column to optional.- Parameters:
name
- name of the column to mark optional- Returns:
- this for method chaining
-
requireColumn
UpdateSchema requireColumn(java.lang.String name)
Update a column to required.This is an incompatible change that can break reading older data. This method will result in an exception unless
allowIncompatibleChanges()
has been called.- Parameters:
name
- name of the column to mark required- Returns:
- this for method chaining
-
deleteColumn
UpdateSchema deleteColumn(java.lang.String name)
Delete a column in the schema.The name is used to find the column to delete using
Schema.findField(String)
.- Parameters:
name
- name of the column to delete- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name doesn't identify a column in the schema or if this change conflicts with other additions, renames, or updates.
-
moveFirst
UpdateSchema moveFirst(java.lang.String name)
Move a column from its current position to the start of the schema or its parent struct.- Parameters:
name
- name of the column to move- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name doesn't identify a column in the schema or if this change conflicts with other changes.
-
moveBefore
UpdateSchema moveBefore(java.lang.String name, java.lang.String beforeName)
Move a column from its current position to directly before a reference column.The name is used to find the column to move using
Schema.findField(String)
. If the name identifies a nested column, it can only be moved within the nested struct that contains it.- Parameters:
name
- name of the column to movebeforeName
- name of the reference column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name doesn't identify a column in the schema or if this change conflicts with other changes.
-
moveAfter
UpdateSchema moveAfter(java.lang.String name, java.lang.String afterName)
Move a column from its current position to directly after a reference column.The name is used to find the column to move using
Schema.findField(String)
. If the name identifies a nested column, it can only be moved within the nested struct that contains it.- Parameters:
name
- name of the column to moveafterName
- name of the reference column- Returns:
- this for method chaining
- Throws:
java.lang.IllegalArgumentException
- If name doesn't identify a column in the schema or if this change conflicts with other changes.
-
unionByNameWith
UpdateSchema unionByNameWith(Schema newSchema)
Applies all field additions and updates from the provided new schema to the existing schema so to create a union schema.For fields with same canonical names in both schemas it is required that the widen types is supported using
updateColumn(String, Type.PrimitiveType)
Only supports turning a previously required field into an optional one if it is marked optional in the provided new schema using
makeColumnOptional(String)
Only supports updating existing field docs with fields docs from the provided new schema using
updateColumnDoc(String, String)
- Parameters:
newSchema
- a schema used in conjunction with the existing schema to create a union schema- Returns:
- this for method chaining
- Throws:
java.lang.IllegalStateException
- If it encounters errors during provided schema traversaljava.lang.IllegalArgumentException
- If name doesn't identify a column in the schema or if this change introduces a type incompatibility or if it conflicts with other additions, renames, or updates.
-
setIdentifierFields
UpdateSchema setIdentifierFields(java.util.Collection<java.lang.String> names)
Set the identifier fields given a set of field names.Because identifier fields are unique, duplicated names will be ignored. See
Schema.identifierFieldIds()
to learn more about Iceberg identifier.- Parameters:
names
- names of the columns to set as identifier fields- Returns:
- this for method chaining
-
setIdentifierFields
default UpdateSchema setIdentifierFields(java.lang.String... names)
Set the identifier fields given some field names. SeesetIdentifierFields(Collection)
for more details.- Parameters:
names
- names of the columns to set as identifier fields- Returns:
- this for method chaining
-
caseSensitive
default UpdateSchema caseSensitive(boolean caseSensitive)
Determines if the case of schema needs to be considered when comparing column names- Parameters:
caseSensitive
- when false case is not considered in column name comparisons.- Returns:
- this for method chaining
-
-