Package org.apache.iceberg.formats

Interface FormatModel<D,S>

Type Parameters:
D - output type used for reading data, and input type for writing data and deletes
S - the type of the schema for the input/output data
All Known Implementing Classes:
AvroFormatModel, BaseFormatModel, ORCFormatModel, ParquetFormatModel
public interface FormatModel<D,S>
Interface that provides a unified abstraction for converting between data file formats and input/output data representations.

FormatModel serves as a bridge between storage formats (FileFormat) and expected input/output data structures, optimizing performance through direct conversion without intermediate representations. File format implementations handle the low-level parsing details while the object model determines the in-memory representation used for the parsed data. Together, these provide a consistent API for consuming data files while optimizing for specific processing engines.

Iceberg provides some built-in object models and processing engines can implement custom object models to integrate with Iceberg's file reading and writing capabilities.

  • Method Summary

    Modifier and Type
    Method
    Description
    FileFormat
    format()
    The file format which is read/written by the object model.
    ReadBuilder<D,S>
    readBuilder(InputFile inputFile)
    Creates a file reader builder for the specified input file.
    Class<S>
    schemaType()
    Return the schema type class for the object model implementation processed by this factory.
    Class<? extends D>
    type()
    Return the row type class for the object model implementation processed by this factory.
    ModelWriteBuilder<D,S>
    writeBuilder(EncryptedOutputFile outputFile)
    Creates a writer builder for data files.

  • Method Details

    • format

      FileFormat format()
      The file format which is read/written by the object model.

    • type

      Class<? extends D> type()
      Return the row type class for the object model implementation processed by this factory.

      The model types act as a contract specifying the expected data structures for both reading (converting file formats into output objects) and writing (converting input objects into file formats). This ensures proper integration between Iceberg's storage layer and processing engines.

      Processing engines can define their own object models by implementing this interface and using their own model name. They can register these models with Iceberg by using the FormatModelRegistry. This allows custom data representations to be seamlessly integrated with Iceberg's file format handlers.

      Returns:
      the type of the data structures handled by this model implementation

    • schemaType

      Class<S> schemaType()
      Return the schema type class for the object model implementation processed by this factory.
      Returns:
      the type of the schema for the data structures handled by this model implementation

    • writeBuilder

      ModelWriteBuilder<D,S> writeBuilder(EncryptedOutputFile outputFile)
      Creates a writer builder for data files.

      The returned ModelWriteBuilder configures and creates a writer that converts input objects into the file format supported by this factory.

      Parameters:
      outputFile - destination for the written data
      Returns:
      configured writer builder

    • readBuilder

      ReadBuilder<D,S> readBuilder(InputFile inputFile)
      Creates a file reader builder for the specified input file.

      The returned ReadBuilder configures and creates a reader that converts data from the file format into output objects supported by this factory.

      Parameters:
      inputFile - source file to read from
      Returns:
      configured reader builder for the specified input