Package org.apache.iceberg

Class Schema

java.lang.Object
org.apache.iceberg.Schema
All Implemented Interfaces:
Serializable
public class Schema extends Object implements Serializable
The schema of a data table.

Schema ID will only be populated when reading from/writing to table metadata, otherwise it will be default to 0.

See Also:

  • Constructor Details

  • Method Details

    • schemaId

      public int schemaId()
      Returns the schema ID for this schema.

      Note that schema ID will only be populated when reading from/writing to table metadata, otherwise it will be default to 0.

    • highestFieldId

      public int highestFieldId()
      Returns the highest field ID in this schema, including nested fields.

    • getAliases

      public Map<String,Integer> getAliases()
      Returns an alias map for this schema, if set.

      Alias maps are created when translating an external schema, like an Avro Schema, to this format. The original column names can be provided in a Map when constructing this Schema.

      Returns:
      a Map of column aliases to field ids

    • idToName

      public Map<Integer,String> idToName()
      Returns a map for this schema between field id and qualified field names.
      Returns:
      a map of field id to qualified field names

    • asStruct

      public Types.StructType asStruct()
      Returns the underlying struct type for this schema.
      Returns:
      the StructType version of this schema.

    • columns

      public List<Types.NestedField> columns()
      Returns a List of the columns in this Schema.

    • identifierFieldIds

      public Set<Integer> identifierFieldIds()
      The set of identifier field IDs.

      Identifier is a concept similar to primary key in a relational database system. It consists of a unique set of primitive fields in the schema. An identifier field must be at root, or nested in a chain of structs (no maps or lists). A row should be unique in a table based on the values of the identifier fields. Optional, float and double columns cannot be used as identifier fields. However, Iceberg identifier differs from primary key in the following ways:

      • Iceberg does not enforce the uniqueness of a row based on this identifier information. It is used for operations like upsert to define the default upsert key.
      • A nested field in a struct can be used as an identifier. For example, if there is a "last_name" field inside a "user" struct in a schema, field "user.last_name" can be set as a part of the identifier field.

      Returns:
      the set of identifier field IDs in this schema.

    • identifierFieldNames

      public Set<String> identifierFieldNames()
      Returns the set of identifier field names.

    • findType

      public Type findType(String name)
      Returns the Type of a sub-field identified by the field name.
      Parameters:
      name - a field name
      Returns:
      a Type for the sub-field or null if it is not found

    • findType

      public Type findType(int id)
      Returns the Type of a sub-field identified by the field id.
      Parameters:
      id - a field id
      Returns:
      a Type for the sub-field or null if it is not found

    • findField

      public Types.NestedField findField(int id)
      Returns the sub-field identified by the field id as a Types.NestedField.
      Parameters:
      id - a field id
      Returns:
      the sub-field or null if it is not found

    • findField

      public Types.NestedField findField(String name)
      Returns a sub-field by name as a Types.NestedField.

      The result may be a top-level or a nested field.

      Parameters:
      name - a String name
      Returns:
      a Type for the sub-field or null if it is not found

    • caseInsensitiveFindField

      public Types.NestedField caseInsensitiveFindField(String name)
      Returns a sub-field by name as a Types.NestedField.

      The result may be a top-level or a nested field.

      Parameters:
      name - a String name
      Returns:
      the sub-field or null if it is not found

    • findColumnName

      public String findColumnName(int id)
      Returns the full column name for the given id.
      Parameters:
      id - a field id
      Returns:
      the full column name in this schema that resolves to the id

    • aliasToId

      public Integer aliasToId(String alias)
      Returns the column id for the given column alias. Column aliases are set by conversions from Parquet or Avro to this Schema type.
      Parameters:
      alias - a full column name in the unconverted data schema
      Returns:
      the column id in this schema, or null if the column wasn't found

    • idToAlias

      public String idToAlias(Integer fieldId)
      Returns the full column name in the unconverted data schema for the given column id. Column aliases are set by conversions from Parquet or Avro to this Schema type.
      Parameters:
      fieldId - a column id in this schema
      Returns:
      the full column name in the unconverted data schema, or null if one wasn't found

    • accessorForField

      public Accessor<StructLike> accessorForField(int id)
      Returns an accessor for retrieving the data from StructLike.

      Accessors do not retrieve data contained in lists or maps.

      Parameters:
      id - a column id in this schema
      Returns:
      an Accessor to retrieve values from a StructLike row

    • select

      public Schema select(String... names)
      Creates a projection schema for a subset of columns, selected by name.

      Names that identify nested fields will select part or all of the field's top-level column.

      Parameters:
      names - String names for selected columns
      Returns:
      a projection schema from this schema, by name

    • select

      public Schema select(Collection<String> names)
      Creates a projection schema for a subset of columns, selected by name.

      Names that identify nested fields will select part or all of the field's top-level column.

      Parameters:
      names - a List of String names for selected columns
      Returns:
      a projection schema from this schema, by name

    • caseInsensitiveSelect

      public Schema caseInsensitiveSelect(String... names)
      Creates a projection schema for a subset of columns, selected by case insensitive names

      Names that identify nested fields will select part or all of the field's top-level column.

      Parameters:
      names - a List of String names for selected columns
      Returns:
      a projection schema from this schema, by names

    • caseInsensitiveSelect

      public Schema caseInsensitiveSelect(Collection<String> names)
      Creates a projection schema for a subset of columns, selected by case insensitive names

      Names that identify nested fields will select part or all of the field's top-level column.

      Parameters:
      names - a List of String names for selected columns
      Returns:
      a projection schema from this schema, by names

    • sameSchema

      public boolean sameSchema(Schema anotherSchema)
      Checks whether this schema is equivalent to another schema while ignoring the schema ID.
      Parameters:
      anotherSchema - another schema
      Returns:
      true if this schema is equivalent to the given schema

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • idsToReassigned

      public Map<Integer,Integer> idsToReassigned()
      The ID's of some fields will be re-assigned if GetID is specified for the Schema.
      Returns:
      map of original to reassigned field ids

    • idsToOriginal

      public Map<Integer,Integer> idsToOriginal()
      The ID's of some fields will be re-assigned if GetID is specified for the Schema.
      Returns:
      map of reassigned to original field ids

    • checkCompatibility

      public static void checkCompatibility(Schema schema, int formatVersion)
      Check the compatibility of the schema with a format version.

      This validates that the schema does not contain types that were released in later format versions.

      Parameters:
      schema - a Schema
      formatVersion - table format version