org.apache.iceberg.gcp.gcs

Class GCSFileIO

java.lang.Object

org.apache.iceberg.gcp.gcs.GCSFileIO

All Implemented Interfaces:

java.io.Closeable, java.io.Serializable, java.lang.AutoCloseable, FileIO

public class GCSFileIO
extends java.lang.Object
implements FileIO

FileIO Implementation backed by Google Cloud Storage (GCS)

Locations follow the conventions used by BlobId.fromGsUtilUri that follow the convention

 gs://<bucket>/<blob_path>

See Cloud Storage Overview

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
GCSFileIO() No-arg constructor to load the FileIO dynamically.
GCSFileIO(SerializableSupplier<com.google.cloud.storage.Storage> storageSupplier, GCPProperties gcpProperties) Constructor with custom storage supplier and GCP properties.

Method Summary

Modifier and Type	Method and Description
void	close() Close File IO to release underlying resources.
void	deleteFile(java.lang.String path) Delete the file at the given path.
void	initialize(java.util.Map<java.lang.String,java.lang.String> props) Initialize File IO from catalog properties.
InputFile	newInputFile(java.lang.String path) Get a InputFile instance to read bytes from the file at the given path.
InputFile	newInputFile(java.lang.String path, long length) Get a InputFile instance to read bytes from the file at the given path, with a known file length.
OutputFile	newOutputFile(java.lang.String path) Get a OutputFile instance to write bytes to the file at the given path.
java.util.Map<java.lang.String,java.lang.String>	properties() Returns the property map used to configure this FileIO

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.io.FileIO

deleteFile, deleteFile

Constructor Detail

GCSFileIO

public GCSFileIO()

No-arg constructor to load the FileIO dynamically.

All fields are initialized by calling initialize(Map) later.

GCSFileIO

public GCSFileIO(SerializableSupplier<com.google.cloud.storage.Storage> storageSupplier,
                 GCPProperties gcpProperties)

Constructor with custom storage supplier and GCP properties.

Calling initialize(Map) will overwrite information set in this constructor.

Parameters:

storageSupplier - storage supplier

gcpProperties - gcp properties

Method Detail

newInputFile

public InputFile newInputFile(java.lang.String path)

Description copied from interface: FileIO

Get a InputFile instance to read bytes from the file at the given path.

Specified by:

newInputFile in interface FileIO

newInputFile

public InputFile newInputFile(java.lang.String path,
                              long length)

Description copied from interface: FileIO

Get a InputFile instance to read bytes from the file at the given path, with a known file length.

Specified by:

newInputFile in interface FileIO

newOutputFile

public OutputFile newOutputFile(java.lang.String path)

Description copied from interface: FileIO

Get a OutputFile instance to write bytes to the file at the given path.

Specified by:

newOutputFile in interface FileIO

deleteFile

public void deleteFile(java.lang.String path)

Description copied from interface: FileIO

Delete the file at the given path.

Specified by:

deleteFile in interface FileIO

properties

public java.util.Map<java.lang.String,java.lang.String> properties()

Description copied from interface: FileIO

Returns the property map used to configure this FileIO

Specified by:

properties in interface FileIO

initialize

public void initialize(java.util.Map<java.lang.String,java.lang.String> props)

Description copied from interface: FileIO

Initialize File IO from catalog properties.

Specified by:

initialize in interface FileIO

Parameters:

props - catalog properties

close

public void close()

Description copied from interface: FileIO

Close File IO to release underlying resources.

Calling this method is only required when this FileIO instance is no longer expected to be used, and the resources it holds need to be explicitly released to avoid resource leaks.

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface FileIO