Package org.apache.iceberg.aws.s3

Class S3FileIO

  • All Implemented Interfaces:
    java.io.Closeable, java.io.Serializable, java.lang.AutoCloseable, FileIO
    public class S3FileIO
extends java.lang.Object
implements FileIO
    FileIO implementation backed by S3.

    Locations used must follow the conventions for S3 URIs (e.g. s3://bucket/path...). URIs with schemes s3a, s3n, https are also treated as s3 file paths. Using this FileIO with other schemes will result in ValidationException.

    See Also:
    Serialized Form

    • Constructor Summary

      Constructors 
      Constructor Description
      S3FileIO()
      No-arg constructor to load the FileIO dynamically.
      S3FileIO​(SerializableSupplier<software.amazon.awssdk.services.s3.S3Client> s3)
      Constructor with custom s3 supplier and default AWS properties.
      S3FileIO​(SerializableSupplier<software.amazon.awssdk.services.s3.S3Client> s3, AwsProperties awsProperties)
      Constructor with custom s3 supplier and AWS properties.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()
      Close File IO to release underlying resources.
      void deleteFile​(java.lang.String path)
      Delete the file at the given path.
      void initialize​(java.util.Map<java.lang.String,​java.lang.String> properties)
      Initialize File IO from catalog properties.
      InputFile newInputFile​(java.lang.String path)
      Get a InputFile instance to read bytes from the file at the given path.
      OutputFile newOutputFile​(java.lang.String path)
      Get a OutputFile instance to write bytes to the file at the given path.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • S3FileIO

        public S3FileIO()
        No-arg constructor to load the FileIO dynamically.

        All fields are initialized by calling initialize(Map) later.

      • S3FileIO

        public S3FileIO​(SerializableSupplier<software.amazon.awssdk.services.s3.S3Client> s3)
        Constructor with custom s3 supplier and default AWS properties.

        Calling initialize(Map) will overwrite information set in this constructor.

        Parameters:
        s3 - s3 supplier

      • S3FileIO

        public S3FileIO​(SerializableSupplier<software.amazon.awssdk.services.s3.S3Client> s3,
                AwsProperties awsProperties)
        Constructor with custom s3 supplier and AWS properties.

        Calling initialize(Map) will overwrite information set in this constructor.

        Parameters:
        s3 - s3 supplier
        awsProperties - aws properties

    • Method Detail

      • newInputFile

        public InputFile newInputFile​(java.lang.String path)
        Description copied from interface: FileIO
        Get a InputFile instance to read bytes from the file at the given path.
        Specified by:
        newInputFile in interface FileIO

      • newOutputFile

        public OutputFile newOutputFile​(java.lang.String path)
        Description copied from interface: FileIO
        Get a OutputFile instance to write bytes to the file at the given path.
        Specified by:
        newOutputFile in interface FileIO

      • deleteFile

        public void deleteFile​(java.lang.String path)
        Description copied from interface: FileIO
        Delete the file at the given path.
        Specified by:
        deleteFile in interface FileIO

      • initialize

        public void initialize​(java.util.Map<java.lang.String,​java.lang.String> properties)
        Description copied from interface: FileIO
        Initialize File IO from catalog properties.
        Specified by:
        initialize in interface FileIO
        Parameters:
        properties - catalog properties

      • close

        public void close()
        Description copied from interface: FileIO
        Close File IO to release underlying resources.

        Calling this method is only required when this FileIO instance is no longer expected to be used, and the resources it holds need to be explicitly released to avoid resource leaks.

        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Specified by:
        close in interface FileIO