Package org.apache.iceberg.io

Class ResolvingFileIO

  • All Implemented Interfaces:
    java.io.Closeable, java.io.Serializable, java.lang.AutoCloseable, org.apache.hadoop.conf.Configurable, HadoopConfigurable, FileIO
    public class ResolvingFileIO
extends java.lang.Object
implements FileIO, HadoopConfigurable
    FileIO implementation that uses location scheme to choose the correct FileIO implementation.
    See Also:
    Serialized Form

    • Constructor Summary

      Constructors 
      Constructor Description
      ResolvingFileIO()
      No-arg constructor to load the FileIO dynamically.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()
      Close File IO to release underlying resources.
      void deleteFile​(java.lang.String location)
      Delete the file at the given path.
      org.apache.hadoop.conf.Configuration getConf()  
      void initialize​(java.util.Map<java.lang.String,​java.lang.String> newProperties)
      Initialize File IO from catalog properties.
      InputFile newInputFile​(java.lang.String location)
      Get a InputFile instance to read bytes from the file at the given path.
      InputFile newInputFile​(java.lang.String location, long length)
      Get a InputFile instance to read bytes from the file at the given path, with a known file length.
      OutputFile newOutputFile​(java.lang.String location)
      Get a OutputFile instance to write bytes to the file at the given path.
      java.util.Map<java.lang.String,​java.lang.String> properties()
      Returns the property map used to configure this FileIO
      void serializeConfWith​(java.util.function.Function<org.apache.hadoop.conf.Configuration,​SerializableSupplier<org.apache.hadoop.conf.Configuration>> confSerializer)
      Take a function that serializes Hadoop configuration into a supplier.
      void setConf​(org.apache.hadoop.conf.Configuration conf)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • ResolvingFileIO

        public ResolvingFileIO()
        No-arg constructor to load the FileIO dynamically.

        All fields are initialized by calling initialize(Map) later.

    • Method Detail

      • newInputFile

        public InputFile newInputFile​(java.lang.String location)
        Description copied from interface: FileIO
        Get a InputFile instance to read bytes from the file at the given path.
        Specified by:
        newInputFile in interface FileIO

      • newInputFile

        public InputFile newInputFile​(java.lang.String location,
                              long length)
        Description copied from interface: FileIO
        Get a InputFile instance to read bytes from the file at the given path, with a known file length.
        Specified by:
        newInputFile in interface FileIO

      • newOutputFile

        public OutputFile newOutputFile​(java.lang.String location)
        Description copied from interface: FileIO
        Get a OutputFile instance to write bytes to the file at the given path.
        Specified by:
        newOutputFile in interface FileIO

      • deleteFile

        public void deleteFile​(java.lang.String location)
        Description copied from interface: FileIO
        Delete the file at the given path.
        Specified by:
        deleteFile in interface FileIO

      • properties

        public java.util.Map<java.lang.String,​java.lang.String> properties()
        Description copied from interface: FileIO
        Returns the property map used to configure this FileIO
        Specified by:
        properties in interface FileIO

      • initialize

        public void initialize​(java.util.Map<java.lang.String,​java.lang.String> newProperties)
        Description copied from interface: FileIO
        Initialize File IO from catalog properties.
        Specified by:
        initialize in interface FileIO
        Parameters:
        newProperties - catalog properties

      • close

        public void close()
        Description copied from interface: FileIO
        Close File IO to release underlying resources.

        Calling this method is only required when this FileIO instance is no longer expected to be used, and the resources it holds need to be explicitly released to avoid resource leaks.

        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Specified by:
        close in interface FileIO

      • serializeConfWith

        public void serializeConfWith​(java.util.function.Function<org.apache.hadoop.conf.Configuration,​SerializableSupplier<org.apache.hadoop.conf.Configuration>> confSerializer)
        Description copied from interface: HadoopConfigurable
        Take a function that serializes Hadoop configuration into a supplier. An implementation is supposed to pass in its current Hadoop configuration into this function, and the result can be safely serialized for future use.
        Specified by:
        serializeConfWith in interface HadoopConfigurable
        Parameters:
        confSerializer - A function that takes Hadoop configuration and returns a serializable supplier of it.

      • setConf

        public void setConf​(org.apache.hadoop.conf.Configuration conf)
        Specified by:
        setConf in interface org.apache.hadoop.conf.Configurable

      • getConf

        public org.apache.hadoop.conf.Configuration getConf()
        Specified by:
        getConf in interface org.apache.hadoop.conf.Configurable