Class HadoopFileIO

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.hadoop.conf.Configuration conf()  
      void deleteFile​(java.lang.String path)
      Delete the file at the given path.
      org.apache.hadoop.conf.Configuration getConf()  
      InputFile newInputFile​(java.lang.String path)
      Get a InputFile instance to read bytes from the file at the given path.
      OutputFile newOutputFile​(java.lang.String path)
      Get a OutputFile instance to write bytes to the file at the given path.
      void serializeConfWith​(java.util.function.Function<org.apache.hadoop.conf.Configuration,​SerializableSupplier<org.apache.hadoop.conf.Configuration>> confSerializer)
      Take a function that serializes Hadoop configuration into a supplier.
      void setConf​(org.apache.hadoop.conf.Configuration conf)  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • HadoopFileIO

        public HadoopFileIO()
        Constructor used for dynamic FileIO loading.

        Hadoop configuration must be set through setConf(Configuration)

      • HadoopFileIO

        public HadoopFileIO​(org.apache.hadoop.conf.Configuration hadoopConf)
      • HadoopFileIO

        public HadoopFileIO​(SerializableSupplier<org.apache.hadoop.conf.Configuration> hadoopConf)
    • Method Detail

      • conf

        public org.apache.hadoop.conf.Configuration conf()
      • newInputFile

        public InputFile newInputFile​(java.lang.String path)
        Description copied from interface: FileIO
        Get a InputFile instance to read bytes from the file at the given path.
        Specified by:
        newInputFile in interface FileIO
      • newOutputFile

        public OutputFile newOutputFile​(java.lang.String path)
        Description copied from interface: FileIO
        Get a OutputFile instance to write bytes to the file at the given path.
        Specified by:
        newOutputFile in interface FileIO
      • deleteFile

        public void deleteFile​(java.lang.String path)
        Description copied from interface: FileIO
        Delete the file at the given path.
        Specified by:
        deleteFile in interface FileIO
      • setConf

        public void setConf​(org.apache.hadoop.conf.Configuration conf)
        Specified by:
        setConf in interface org.apache.hadoop.conf.Configurable
      • getConf

        public org.apache.hadoop.conf.Configuration getConf()
        Specified by:
        getConf in interface org.apache.hadoop.conf.Configurable
      • serializeConfWith

        public void serializeConfWith​(java.util.function.Function<org.apache.hadoop.conf.Configuration,​SerializableSupplier<org.apache.hadoop.conf.Configuration>> confSerializer)
        Description copied from interface: HadoopConfigurable
        Take a function that serializes Hadoop configuration into a supplier. An implementation is supposed to pass in its current Hadoop configuration into this function, and the result can be safely serialized for future use.
        Specified by:
        serializeConfWith in interface HadoopConfigurable
        Parameters:
        confSerializer - A function that takes Hadoop configuration and returns a serializable supplier of it.