Class MapredIcebergInputFormat<T>

  • Type Parameters:
    T - Java class of records constructed by Iceberg; default is Record
    All Implemented Interfaces:
    org.apache.hadoop.mapred.InputFormat<java.lang.Void,​Container<T>>
    Direct Known Subclasses:
    HiveIcebergInputFormat

    public class MapredIcebergInputFormat<T>
    extends java.lang.Object
    implements org.apache.hadoop.mapred.InputFormat<java.lang.Void,​Container<T>>
    Generic MR v1 InputFormat API for Iceberg.
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      static InputFormatConfig.ConfigBuilder configure​(org.apache.hadoop.mapred.JobConf job)
      Configures the JobConf to use the MapredIcebergInputFormat and returns a helper to add further configuration.
      org.apache.hadoop.mapred.RecordReader<java.lang.Void,​Container<T>> getRecordReader​(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)  
      org.apache.hadoop.mapred.InputSplit[] getSplits​(org.apache.hadoop.mapred.JobConf job, int numSplits)  
      static org.apache.hadoop.mapreduce.TaskAttemptContext newTaskAttemptContext​(org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • MapredIcebergInputFormat

        public MapredIcebergInputFormat()
    • Method Detail

      • configure

        public static InputFormatConfig.ConfigBuilder configure​(org.apache.hadoop.mapred.JobConf job)
        Configures the JobConf to use the MapredIcebergInputFormat and returns a helper to add further configuration.
        Parameters:
        job - the JobConf to configure
      • getSplits

        public org.apache.hadoop.mapred.InputSplit[] getSplits​(org.apache.hadoop.mapred.JobConf job,
                                                               int numSplits)
                                                        throws java.io.IOException
        Specified by:
        getSplits in interface org.apache.hadoop.mapred.InputFormat<java.lang.Void,​Container<T>>
        Throws:
        java.io.IOException
      • getRecordReader

        public org.apache.hadoop.mapred.RecordReader<java.lang.Void,​Container<T>> getRecordReader​(org.apache.hadoop.mapred.InputSplit split,
                                                                                                        org.apache.hadoop.mapred.JobConf job,
                                                                                                        org.apache.hadoop.mapred.Reporter reporter)
                                                                                                 throws java.io.IOException
        Specified by:
        getRecordReader in interface org.apache.hadoop.mapred.InputFormat<java.lang.Void,​Container<T>>
        Throws:
        java.io.IOException
      • newTaskAttemptContext

        public static org.apache.hadoop.mapreduce.TaskAttemptContext newTaskAttemptContext​(org.apache.hadoop.mapred.JobConf job,
                                                                                           org.apache.hadoop.mapred.Reporter reporter)