Interface HadoopConfigurable

  • All Superinterfaces:
    org.apache.hadoop.conf.Configurable
    All Known Implementing Classes:
    HadoopFileIO, ResolvingFileIO

    public interface HadoopConfigurable
    extends org.apache.hadoop.conf.Configurable
    An interface that extends the Hadoop Configurable interface to offer better serialization support for customizable Iceberg objects such as FileIO.

    If an object is serialized and needs to use Hadoop configuration, it is recommended for the object to implement this interface so that a serializable supplier of configuration can be provided instead of an actual Hadoop configuration which is not serializable.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      void serializeConfWith​(java.util.function.Function<org.apache.hadoop.conf.Configuration,​SerializableSupplier<org.apache.hadoop.conf.Configuration>> confSerializer)
      Take a function that serializes Hadoop configuration into a supplier.
      • Methods inherited from interface org.apache.hadoop.conf.Configurable

        getConf, setConf
    • Method Detail

      • serializeConfWith

        void serializeConfWith​(java.util.function.Function<org.apache.hadoop.conf.Configuration,​SerializableSupplier<org.apache.hadoop.conf.Configuration>> confSerializer)
        Take a function that serializes Hadoop configuration into a supplier. An implementation is supposed to pass in its current Hadoop configuration into this function, and the result can be safely serialized for future use.
        Parameters:
        confSerializer - A function that takes Hadoop configuration and returns a serializable supplier of it.