Package org.apache.iceberg.hadoop

Interface HadoopConfigurable

All Superinterfaces:
org.apache.hadoop.conf.Configurable
All Known Implementing Classes:
HadoopFileIO, ResolvingFileIO
public interface HadoopConfigurable extends org.apache.hadoop.conf.Configurable
An interface that extends the Hadoop Configurable interface to offer better serialization support for customizable Iceberg objects such as FileIO.

If an object is serialized and needs to use Hadoop configuration, it is recommended for the object to implement this interface so that a serializable supplier of configuration can be provided instead of an actual Hadoop configuration which is not serializable.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    serializeConfWith(Function<org.apache.hadoop.conf.Configuration,SerializableSupplier<org.apache.hadoop.conf.Configuration>> confSerializer)
    Take a function that serializes Hadoop configuration into a supplier.

    Methods inherited from interface org.apache.hadoop.conf.Configurable

    getConf, setConf

  • Method Details

    • serializeConfWith

      void serializeConfWith(Function<org.apache.hadoop.conf.Configuration,SerializableSupplier<org.apache.hadoop.conf.Configuration>> confSerializer)
      Take a function that serializes Hadoop configuration into a supplier. An implementation is supposed to pass in its current Hadoop configuration into this function, and the result can be safely serialized for future use.
      Parameters:
      confSerializer - A function that takes Hadoop configuration and returns a serializable supplier of it.