Class HadoopInputFile

  • All Implemented Interfaces:
    InputFile

    public class HadoopInputFile
    extends java.lang.Object
    implements InputFile
    InputFile implementation using the Hadoop FileSystem API.

    This class is based on Parquet's HadoopInputFile.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      boolean exists()
      Checks whether the file exists.
      static HadoopInputFile fromLocation​(java.lang.CharSequence location, long length, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromLocation​(java.lang.CharSequence location, long length, org.apache.hadoop.fs.FileSystem fs)  
      static HadoopInputFile fromLocation​(java.lang.CharSequence location, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromLocation​(java.lang.CharSequence location, org.apache.hadoop.fs.FileSystem fs)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.fs.FileSystem fs)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs)  
      static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.conf.Configuration conf)  
      static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.FileSystem fs)  
      static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)  
      java.lang.String[] getBlockLocations​(long start, long end)  
      org.apache.hadoop.conf.Configuration getConf()  
      org.apache.hadoop.fs.FileSystem getFileSystem()  
      long getLength()
      Returns the total length of the file, in bytes
      org.apache.hadoop.fs.Path getPath()  
      org.apache.hadoop.fs.FileStatus getStat()  
      java.lang.String location()
      The fully-qualified location of the input file as a String.
      SeekableInputStream newStream()
      Opens a new SeekableInputStream for the underlying data file
      java.lang.String toString()  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Field Detail

      • NO_LOCATION_PREFERENCE

        public static final java.lang.String[] NO_LOCATION_PREFERENCE
    • Method Detail

      • fromLocation

        public static HadoopInputFile fromLocation​(java.lang.CharSequence location,
                                                   org.apache.hadoop.conf.Configuration conf)
      • fromLocation

        public static HadoopInputFile fromLocation​(java.lang.CharSequence location,
                                                   long length,
                                                   org.apache.hadoop.conf.Configuration conf)
      • fromLocation

        public static HadoopInputFile fromLocation​(java.lang.CharSequence location,
                                                   org.apache.hadoop.fs.FileSystem fs)
      • fromLocation

        public static HadoopInputFile fromLocation​(java.lang.CharSequence location,
                                                   long length,
                                                   org.apache.hadoop.fs.FileSystem fs)
      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                               org.apache.hadoop.conf.Configuration conf)
      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                               long length,
                                               org.apache.hadoop.conf.Configuration conf)
      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                               org.apache.hadoop.fs.FileSystem fs)
      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                               long length,
                                               org.apache.hadoop.fs.FileSystem fs)
      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                               org.apache.hadoop.fs.FileSystem fs,
                                               org.apache.hadoop.conf.Configuration conf)
      • fromPath

        public static HadoopInputFile fromPath​(org.apache.hadoop.fs.Path path,
                                               long length,
                                               org.apache.hadoop.fs.FileSystem fs,
                                               org.apache.hadoop.conf.Configuration conf)
      • fromStatus

        public static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat,
                                                 org.apache.hadoop.conf.Configuration conf)
      • fromStatus

        public static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat,
                                                 org.apache.hadoop.fs.FileSystem fs)
      • fromStatus

        public static HadoopInputFile fromStatus​(org.apache.hadoop.fs.FileStatus stat,
                                                 org.apache.hadoop.fs.FileSystem fs,
                                                 org.apache.hadoop.conf.Configuration conf)
      • getLength

        public long getLength()
        Description copied from interface: InputFile
        Returns the total length of the file, in bytes
        Specified by:
        getLength in interface InputFile
        Returns:
        the total length of the file, in bytes
      • getConf

        public org.apache.hadoop.conf.Configuration getConf()
      • getFileSystem

        public org.apache.hadoop.fs.FileSystem getFileSystem()
      • getStat

        public org.apache.hadoop.fs.FileStatus getStat()
      • getPath

        public org.apache.hadoop.fs.Path getPath()
      • getBlockLocations

        public java.lang.String[] getBlockLocations​(long start,
                                                    long end)
      • location

        public java.lang.String location()
        Description copied from interface: InputFile
        The fully-qualified location of the input file as a String.
        Specified by:
        location in interface InputFile
        Returns:
        the input file location
      • exists

        public boolean exists()
        Description copied from interface: InputFile
        Checks whether the file exists.
        Specified by:
        exists in interface InputFile
        Returns:
        true if the file exists, false otherwise
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object