org.apache.iceberg.mr.mapred

Class MapredIcebergInputFormat<T>

java.lang.Object

org.apache.iceberg.mr.mapred.MapredIcebergInputFormat<T>

Type Parameters:

T - Java class of records constructed by Iceberg; default is Record

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<java.lang.Void,Container<T>>

Direct Known Subclasses:

HiveIcebergInputFormat

public class MapredIcebergInputFormat<T>
extends java.lang.Object
implements org.apache.hadoop.mapred.InputFormat<java.lang.Void,Container<T>>

Generic MR v1 InputFormat API for Iceberg.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MapredIcebergInputFormat.CompatibilityTaskAttemptContextImpl

Constructor Summary

Constructors

Constructor and Description
MapredIcebergInputFormat()

Method Summary

Modifier and Type	Method and Description
static InputFormatConfig.ConfigBuilder	configure(org.apache.hadoop.mapred.JobConf job) Configures the JobConf to use the MapredIcebergInputFormat and returns a helper to add further configuration.
org.apache.hadoop.mapred.RecordReader<java.lang.Void,Container<T>>	getRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)
org.apache.hadoop.mapred.InputSplit[]	getSplits(org.apache.hadoop.mapred.JobConf job, int numSplits)
static org.apache.hadoop.mapreduce.TaskAttemptContext	newTaskAttemptContext(org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapredIcebergInputFormat

public MapredIcebergInputFormat()

Method Detail

configure

public static InputFormatConfig.ConfigBuilder configure(org.apache.hadoop.mapred.JobConf job)

Configures the JobConf to use the MapredIcebergInputFormat and returns a helper to add further configuration.

Parameters:

job - the JobConf to configure

getSplits

public org.apache.hadoop.mapred.InputSplit[] getSplits(org.apache.hadoop.mapred.JobConf job,
                                                       int numSplits)
                                                throws java.io.IOException

Specified by:

getSplits in interface org.apache.hadoop.mapred.InputFormat<java.lang.Void,Container<T>>

Throws:

java.io.IOException

getRecordReader

public org.apache.hadoop.mapred.RecordReader<java.lang.Void,Container<T>> getRecordReader(org.apache.hadoop.mapred.InputSplit split,
                                                                                          org.apache.hadoop.mapred.JobConf job,
                                                                                          org.apache.hadoop.mapred.Reporter reporter)
                                                                                   throws java.io.IOException

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<java.lang.Void,Container<T>>

Throws:

java.io.IOException

newTaskAttemptContext

public static org.apache.hadoop.mapreduce.TaskAttemptContext newTaskAttemptContext(org.apache.hadoop.mapred.JobConf job,
                                                                                   org.apache.hadoop.mapred.Reporter reporter)