org.apache.iceberg.hadoop

Interface HadoopConfigurable

All Superinterfaces:

org.apache.hadoop.conf.Configurable

All Known Implementing Classes:

HadoopFileIO, ResolvingFileIO

public interface HadoopConfigurable
extends org.apache.hadoop.conf.Configurable

An interface that extends the Hadoop Configurable interface to offer better serialization support for customizable Iceberg objects such as FileIO.

If an object is serialized and needs to use Hadoop configuration, it is recommended for the object to implement this interface so that a serializable supplier of configuration can be provided instead of an actual Hadoop configuration which is not serializable.

Method Summary

Modifier and Type	Method and Description
void	serializeConfWith(java.util.function.Function<org.apache.hadoop.conf.Configuration,SerializableSupplier<org.apache.hadoop.conf.Configuration>> confSerializer) Take a function that serializes Hadoop configuration into a supplier.

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Method Detail

serializeConfWith

void serializeConfWith(java.util.function.Function<org.apache.hadoop.conf.Configuration,SerializableSupplier<org.apache.hadoop.conf.Configuration>> confSerializer)

Take a function that serializes Hadoop configuration into a supplier. An implementation is supposed to pass in its current Hadoop configuration into this function, and the result can be safely serialized for future use.

Parameters:

confSerializer - A function that takes Hadoop configuration and returns a serializable supplier of it.