Package org.apache.iceberg.hadoop
Class HadoopInputFile
- java.lang.Object
-
- org.apache.iceberg.hadoop.HadoopInputFile
-
- All Implemented Interfaces:
NativelyEncryptedFile
,InputFile
public class HadoopInputFile extends java.lang.Object implements InputFile, NativelyEncryptedFile
InputFile
implementation using the HadoopFileSystem
API.This class is based on Parquet's HadoopInputFile.
-
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String[]
NO_LOCATION_PREFERENCE
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description boolean
exists()
Checks whether the file exists.static HadoopInputFile
fromLocation(java.lang.CharSequence location, long length, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile
fromLocation(java.lang.CharSequence location, long length, org.apache.hadoop.fs.FileSystem fs)
static HadoopInputFile
fromLocation(java.lang.CharSequence location, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile
fromLocation(java.lang.CharSequence location, org.apache.hadoop.fs.FileSystem fs)
static HadoopInputFile
fromPath(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile
fromPath(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.fs.FileSystem fs)
static HadoopInputFile
fromPath(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile
fromPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile
fromPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs)
static HadoopInputFile
fromPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile
fromStatus(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile
fromStatus(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.FileSystem fs)
static HadoopInputFile
fromStatus(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)
java.lang.String[]
getBlockLocations(long start, long end)
org.apache.hadoop.conf.Configuration
getConf()
org.apache.hadoop.fs.FileSystem
getFileSystem()
long
getLength()
Returns the total length of the file, in bytesorg.apache.hadoop.fs.Path
getPath()
org.apache.hadoop.fs.FileStatus
getStat()
java.lang.String
location()
The fully-qualified location of the input file as a String.NativeFileCryptoParameters
nativeCryptoParameters()
SeekableInputStream
newStream()
Opens a newSeekableInputStream
for the underlying data filevoid
setNativeCryptoParameters(NativeFileCryptoParameters nativeCryptoParameters)
java.lang.String
toString()
-
-
-
Method Detail
-
fromLocation
public static HadoopInputFile fromLocation(java.lang.CharSequence location, org.apache.hadoop.conf.Configuration conf)
-
fromLocation
public static HadoopInputFile fromLocation(java.lang.CharSequence location, long length, org.apache.hadoop.conf.Configuration conf)
-
fromLocation
public static HadoopInputFile fromLocation(java.lang.CharSequence location, org.apache.hadoop.fs.FileSystem fs)
-
fromLocation
public static HadoopInputFile fromLocation(java.lang.CharSequence location, long length, org.apache.hadoop.fs.FileSystem fs)
-
fromPath
public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf)
-
fromPath
public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.conf.Configuration conf)
-
fromPath
public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs)
-
fromPath
public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.fs.FileSystem fs)
-
fromPath
public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)
-
fromPath
public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)
-
fromStatus
public static HadoopInputFile fromStatus(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.conf.Configuration conf)
-
fromStatus
public static HadoopInputFile fromStatus(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.FileSystem fs)
-
fromStatus
public static HadoopInputFile fromStatus(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)
-
getLength
public long getLength()
Description copied from interface:InputFile
Returns the total length of the file, in bytes
-
newStream
public SeekableInputStream newStream()
Description copied from interface:InputFile
Opens a newSeekableInputStream
for the underlying data file
-
getConf
public org.apache.hadoop.conf.Configuration getConf()
-
getFileSystem
public org.apache.hadoop.fs.FileSystem getFileSystem()
-
getStat
public org.apache.hadoop.fs.FileStatus getStat()
-
getPath
public org.apache.hadoop.fs.Path getPath()
-
getBlockLocations
public java.lang.String[] getBlockLocations(long start, long end)
-
location
public java.lang.String location()
Description copied from interface:InputFile
The fully-qualified location of the input file as a String.
-
exists
public boolean exists()
Description copied from interface:InputFile
Checks whether the file exists.
-
nativeCryptoParameters
public NativeFileCryptoParameters nativeCryptoParameters()
- Specified by:
nativeCryptoParameters
in interfaceNativelyEncryptedFile
-
setNativeCryptoParameters
public void setNativeCryptoParameters(NativeFileCryptoParameters nativeCryptoParameters)
- Specified by:
setNativeCryptoParameters
in interfaceNativelyEncryptedFile
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
-