Class Schema

  • All Implemented Interfaces:
    java.io.Serializable

    public class Schema
    extends java.lang.Object
    implements java.io.Serializable
    The schema of a data table.
    See Also:
    Serialized Form
    • Constructor Detail

      • Schema

        public Schema​(java.util.List<Types.NestedField> columns,
                      java.util.Map<java.lang.String,​java.lang.Integer> aliases)
    • Method Detail

      • getAliases

        public java.util.Map<java.lang.String,​java.lang.Integer> getAliases()
        Returns an alias map for this schema, if set.

        Alias maps are created when translating an external schema, like an Avro Schema, to this format. The original column names can be provided in a Map when constructing this Schema.

        Returns:
        a Map of column aliases to field ids
      • asStruct

        public Types.StructType asStruct()
        Returns the underlying struct type for this schema.
        Returns:
        the StructType version of this schema.
      • findType

        public Type findType​(java.lang.String name)
        Returns the Type of a sub-field identified by the field name.
        Parameters:
        name - a field name
        Returns:
        a Type for the sub-field or null if it is not found
      • findType

        public Type findType​(int id)
        Returns the Type of a sub-field identified by the field id.
        Parameters:
        id - a field id
        Returns:
        a Type for the sub-field or null if it is not found
      • findField

        public Types.NestedField findField​(int id)
        Returns the sub-field identified by the field id as a Types.NestedField.
        Parameters:
        id - a field id
        Returns:
        the sub-field or null if it is not found
      • findField

        public Types.NestedField findField​(java.lang.String name)
        Returns a sub-field by name as a Types.NestedField.

        The result may be a top-level or a nested field.

        Parameters:
        name - a String name
        Returns:
        a Type for the sub-field or null if it is not found
      • caseInsensitiveFindField

        public Types.NestedField caseInsensitiveFindField​(java.lang.String name)
        Returns a sub-field by name as a Types.NestedField.

        The result may be a top-level or a nested field.

        Parameters:
        name - a String name
        Returns:
        the sub-field or null if it is not found
      • findColumnName

        public java.lang.String findColumnName​(int id)
        Returns the full column name for the given id.
        Parameters:
        id - a field id
        Returns:
        the full column name in this schema that resolves to the id
      • aliasToId

        public java.lang.Integer aliasToId​(java.lang.String alias)
        Returns the column id for the given column alias. Column aliases are set by conversions from Parquet or Avro to this Schema type.
        Parameters:
        alias - a full column name in the unconverted data schema
        Returns:
        the column id in this schema, or null if the column wasn't found
      • idToAlias

        public java.lang.String idToAlias​(java.lang.Integer fieldId)
        Returns the full column name in the unconverted data schema for the given column id. Column aliases are set by conversions from Parquet or Avro to this Schema type.
        Parameters:
        fieldId - a column id in this schema
        Returns:
        the full column name in the unconverted data schema, or null if one wasn't found
      • accessorForField

        public Accessor<StructLike> accessorForField​(int id)
        Returns an accessor for retrieving the data from StructLike.

        Accessors do not retrieve data contained in lists or maps.

        Parameters:
        id - a column id in this schema
        Returns:
        an Accessor to retrieve values from a StructLike row
      • select

        public Schema select​(java.lang.String... names)
        Creates a projection schema for a subset of columns, selected by name.

        Names that identify nested fields will select part or all of the field's top-level column.

        Parameters:
        names - String names for selected columns
        Returns:
        a projection schema from this schema, by name
      • select

        public Schema select​(java.util.Collection<java.lang.String> names)
        Creates a projection schema for a subset of columns, selected by name.

        Names that identify nested fields will select part or all of the field's top-level column.

        Parameters:
        names - a List of String names for selected columns
        Returns:
        a projection schema from this schema, by name
      • caseInsensitiveSelect

        public Schema caseInsensitiveSelect​(java.lang.String... names)
        Creates a projection schema for a subset of columns, selected by case insensitive names

        Names that identify nested fields will select part or all of the field's top-level column.

        Parameters:
        names - a List of String names for selected columns
        Returns:
        a projection schema from this schema, by names
      • caseInsensitiveSelect

        public Schema caseInsensitiveSelect​(java.util.Collection<java.lang.String> names)
        Creates a projection schema for a subset of columns, selected by case insensitive names

        Names that identify nested fields will select part or all of the field's top-level column.

        Parameters:
        names - a List of String names for selected columns
        Returns:
        a projection schema from this schema, by names
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object