Package org.apache.iceberg.hadoop

Class HadoopInputFile

  • All Implemented Interfaces:
    InputFile
    public class HadoopInputFile
extends java.lang.Object
implements InputFile
    InputFile implementation using the Hadoop FileSystem API.

    This class is based on Parquet's HadoopInputFile.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      boolean exists()
      Checks whether the file exists.
      static HadoopInputFile fromLocation​(java.lang.CharSequence location, long length, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromLocation​(java.lang.CharSequence location, long length, org.apache.hadoop.fs.FileSystem fs)  
      static HadoopInputFile fromLocation​(java.lang.CharSequence location, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromLocation​(java.lang.CharSequence location, org.apache.hadoop.fs.FileSystem fs)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.fs.FileSystem fs)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.FileSystem fs)  
      static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)  
      java.lang.String[] getBlockLocations​(long start, long end)  
      org.apache.hadoop.conf.Configuration getConf()  
      org.apache.hadoop.fs.FileSystem getFileSystem()  
      long getLength()
      Returns the total length of the file, in bytes
      org.apache.hadoop.fs.Path getPath()  
      org.apache.hadoop.fs.FileStatus getStat()  
      java.lang.String location()
      The fully-qualified location of the input file as a String.
      SeekableInputStream newStream()
      Opens a new SeekableInputStream for the underlying data file
      java.lang.String toString()  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Field Detail

      • NO_LOCATION_PREFERENCE

        public static final java.lang.String[] NO_LOCATION_PREFERENCE

    • Method Detail

      • fromLocation

        public static HadoopInputFile fromLocation​(java.lang.CharSequence location,
                                           org.apache.hadoop.conf.Configuration conf)

      • fromLocation

        public static HadoopInputFile fromLocation​(java.lang.CharSequence location,
                                           long length,
                                           org.apache.hadoop.conf.Configuration conf)

      • fromLocation

        public static HadoopInputFile fromLocation​(java.lang.CharSequence location,
                                           org.apache.hadoop.fs.FileSystem fs)

      • fromLocation

        public static HadoopInputFile fromLocation​(java.lang.CharSequence location,
                                           long length,
                                           org.apache.hadoop.fs.FileSystem fs)

      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                       org.apache.hadoop.conf.Configuration conf)

      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                       long length,
                                       org.apache.hadoop.conf.Configuration conf)

      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                       org.apache.hadoop.fs.FileSystem fs)

      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                       long length,
                                       org.apache.hadoop.fs.FileSystem fs)

      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                       org.apache.hadoop.fs.FileSystem fs,
                                       org.apache.hadoop.conf.Configuration conf)

      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                       long length,
                                       org.apache.hadoop.fs.FileSystem fs,
                                       org.apache.hadoop.conf.Configuration conf)

      • fromStatus

        public static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat,
                                         org.apache.hadoop.conf.Configuration conf)

      • fromStatus

        public static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat,
                                         org.apache.hadoop.fs.FileSystem fs)

      • fromStatus

        public static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat,
                                         org.apache.hadoop.fs.FileSystem fs,
                                         org.apache.hadoop.conf.Configuration conf)

      • getLength

        public long getLength()
        Description copied from interface: InputFile
        Returns the total length of the file, in bytes
        Specified by:
        getLength in interface InputFile
        Returns:
        the total length of the file, in bytes

      • getConf

        public org.apache.hadoop.conf.Configuration getConf()

      • getFileSystem

        public org.apache.hadoop.fs.FileSystem getFileSystem()

      • getStat

        public org.apache.hadoop.fs.FileStatus getStat()

      • getPath

        public org.apache.hadoop.fs.Path getPath()

      • getBlockLocations

        public java.lang.String[] getBlockLocations​(long start,
                                            long end)

      • location

        public java.lang.String location()
        Description copied from interface: InputFile
        The fully-qualified location of the input file as a String.
        Specified by:
        location in interface InputFile
        Returns:
        the input file location

      • exists

        public boolean exists()
        Description copied from interface: InputFile
        Checks whether the file exists.
        Specified by:
        exists in interface InputFile
        Returns:
        true if the file exists, false otherwise

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object