Package org.apache.iceberg.aws

Class AwsProperties

  • All Implemented Interfaces:
    java.io.Serializable
    public class AwsProperties
extends java.lang.Object
implements java.io.Serializable
    See Also:
    Serialized Form

    • Field Detail

      • GLUE_CATALOG_ID

        public static final java.lang.String GLUE_CATALOG_ID
        The ID of the Glue Data Catalog where the tables reside. If none is provided, Glue automatically uses the caller's AWS account ID by default.

        For more details, see https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-databases.html

        See Also:
        Constant Field Values

      • GLUE_ACCOUNT_ID

        public static final java.lang.String GLUE_ACCOUNT_ID
        The account ID used in a Glue resource ARN, e.g. arn:aws:glue:us-east-1:1000000000000:table/db1/table1
        See Also:
        Constant Field Values

      • GLUE_CATALOG_SKIP_ARCHIVE

        public static final java.lang.String GLUE_CATALOG_SKIP_ARCHIVE
        If Glue should skip archiving an old table version when creating a new version in a commit. By default Glue archives all old table versions after an UpdateTable call, but Glue has a default max number of archived table versions (can be increased). So for streaming use case with lots of commits, it is recommended to set this value to true.
        See Also:
        Constant Field Values

      • GLUE_CATALOG_SKIP_ARCHIVE_DEFAULT

        public static final boolean GLUE_CATALOG_SKIP_ARCHIVE_DEFAULT
        See Also:
        Constant Field Values

      • GLUE_CATALOG_SKIP_NAME_VALIDATION

        public static final java.lang.String GLUE_CATALOG_SKIP_NAME_VALIDATION
        If Glue should skip name validations It is recommended to stick to Glue best practice in https://docs.aws.amazon.com/athena/latest/ug/glue-best-practices.html to make sure operations are Hive compatible. This is only added for users that have existing conventions using non-standard characters. When database name and table name validation are skipped, there is no guarantee that downstream systems would all support the names.
        See Also:
        Constant Field Values

      • GLUE_CATALOG_SKIP_NAME_VALIDATION_DEFAULT

        public static final boolean GLUE_CATALOG_SKIP_NAME_VALIDATION_DEFAULT
        See Also:
        Constant Field Values

      • GLUE_LAKEFORMATION_ENABLED

        public static final java.lang.String GLUE_LAKEFORMATION_ENABLED
        If set, GlueCatalog will use Lake Formation for access control. For more credential vending details, see: https://docs.aws.amazon.com/lake-formation/latest/dg/api-overview.html. If enabled, the AwsClientFactory implementation must be LakeFormationAwsClientFactory or any class that extends it.
        See Also:
        Constant Field Values

      • GLUE_LAKEFORMATION_ENABLED_DEFAULT

        public static final boolean GLUE_LAKEFORMATION_ENABLED_DEFAULT
        See Also:
        Constant Field Values

      • GLUE_CATALOG_ENDPOINT

        public static final java.lang.String GLUE_CATALOG_ENDPOINT
        Configure an alternative endpoint of the Glue service for GlueCatalog to access.

        This could be used to use GlueCatalog with any glue-compatible metastore service that has a different endpoint

        See Also:
        Constant Field Values

      • DYNAMODB_ENDPOINT

        public static final java.lang.String DYNAMODB_ENDPOINT
        Configure an alternative endpoint of the DynamoDB service to access.
        See Also:
        Constant Field Values

      • DYNAMODB_TABLE_NAME_DEFAULT

        public static final java.lang.String DYNAMODB_TABLE_NAME_DEFAULT
        See Also:
        Constant Field Values

      • CLIENT_ASSUME_ROLE_ARN

        public static final java.lang.String CLIENT_ASSUME_ROLE_ARN
        Used by AssumeRoleAwsClientFactory. If set, all AWS clients will assume a role of the given ARN, instead of using the default credential chain.
        See Also:
        Constant Field Values

      • CLIENT_ASSUME_ROLE_TAGS_PREFIX

        public static final java.lang.String CLIENT_ASSUME_ROLE_TAGS_PREFIX
        Used by AssumeRoleAwsClientFactory to pass a list of sessions. Each session tag consists of a key name and an associated value.
        See Also:
        Constant Field Values

      • CLIENT_ASSUME_ROLE_TIMEOUT_SEC

        public static final java.lang.String CLIENT_ASSUME_ROLE_TIMEOUT_SEC
        Used by AssumeRoleAwsClientFactory. The timeout of the assume role session in seconds, default to 1 hour. At the end of the timeout, a new set of role session credentials will be fetched through a STS client.
        See Also:
        Constant Field Values

      • CLIENT_ASSUME_ROLE_TIMEOUT_SEC_DEFAULT

        public static final int CLIENT_ASSUME_ROLE_TIMEOUT_SEC_DEFAULT
        See Also:
        Constant Field Values

      • CLIENT_ASSUME_ROLE_EXTERNAL_ID

        public static final java.lang.String CLIENT_ASSUME_ROLE_EXTERNAL_ID
        Used by AssumeRoleAwsClientFactory. Optional external ID used to assume an IAM role.

        For more details, see https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html

        See Also:
        Constant Field Values

      • CLIENT_ASSUME_ROLE_REGION

        public static final java.lang.String CLIENT_ASSUME_ROLE_REGION
        Used by AssumeRoleAwsClientFactory. If set, all AWS clients except STS client will use the given region instead of the default region chain.

        The value must be one of Region, such as 'us-east-1'. For more details, see https://docs.aws.amazon.com/general/latest/gr/rande.html

        See Also:
        Constant Field Values

      • CLIENT_ASSUME_ROLE_SESSION_NAME

        public static final java.lang.String CLIENT_ASSUME_ROLE_SESSION_NAME
        Used by AssumeRoleAwsClientFactory. Optional session name used to assume an IAM role.

        For more details, see https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_iam-condition-keys.html#ck_rolesessionname

        See Also:
        Constant Field Values

      • REST_SIGNER_REGION

        public static final java.lang.String REST_SIGNER_REGION
        Region to be used by the SigV4 protocol for signing requests.
        See Also:
        Constant Field Values

      • REST_SIGNING_NAME

        public static final java.lang.String REST_SIGNING_NAME
        The service name to be used by the SigV4 protocol for signing requests.
        See Also:
        Constant Field Values

      • REST_SIGNING_NAME_DEFAULT

        public static final java.lang.String REST_SIGNING_NAME_DEFAULT
        The default service name (API Gateway and lambda) used during SigV4 signing.
        See Also:
        Constant Field Values

      • REST_ACCESS_KEY_ID

        public static final java.lang.String REST_ACCESS_KEY_ID
        Configure the static access key ID used for SigV4 signing.

        When set, the default client factory will use the basic or session credentials provided instead of reading the default credential chain to create S3 access credentials. If REST_SESSION_TOKEN is set, session credential is used, otherwise basic credential is used.

        See Also:
        Constant Field Values

      • REST_SECRET_ACCESS_KEY

        public static final java.lang.String REST_SECRET_ACCESS_KEY
        Configure the static secret access key used for SigV4 signing.

        When set, the default client factory will use the basic or session credentials provided instead of reading the default credential chain to create S3 access credentials. If REST_SESSION_TOKEN is set, session credential is used, otherwise basic credential is used.

        See Also:
        Constant Field Values

      • REST_SESSION_TOKEN

        public static final java.lang.String REST_SESSION_TOKEN
        Configure the static session token used for SigV4.

        When set, the default client factory will use the session credentials provided instead of reading the default credential chain to create access credentials.

        See Also:
        Constant Field Values

    • Constructor Detail

      • AwsProperties

        public AwsProperties()

      • AwsProperties

        public AwsProperties​(java.util.Map<java.lang.String,​java.lang.String> properties)

    • Method Detail

      • stsClientAssumeRoleTags

        public java.util.Set<software.amazon.awssdk.services.sts.model.Tag> stsClientAssumeRoleTags()

      • clientAssumeRoleArn

        public java.lang.String clientAssumeRoleArn()

      • clientAssumeRoleTimeoutSec

        public int clientAssumeRoleTimeoutSec()

      • clientAssumeRoleExternalId

        public java.lang.String clientAssumeRoleExternalId()

      • clientAssumeRoleRegion

        public java.lang.String clientAssumeRoleRegion()

      • clientAssumeRoleSessionName

        public java.lang.String clientAssumeRoleSessionName()

      • glueCatalogId

        public java.lang.String glueCatalogId()

      • setGlueCatalogId

        public void setGlueCatalogId​(java.lang.String id)

      • glueCatalogSkipArchive

        public boolean glueCatalogSkipArchive()

      • setGlueCatalogSkipArchive

        public void setGlueCatalogSkipArchive​(boolean skipArchive)

      • glueCatalogSkipNameValidation

        public boolean glueCatalogSkipNameValidation()

      • setGlueCatalogSkipNameValidation

        public void setGlueCatalogSkipNameValidation​(boolean glueCatalogSkipNameValidation)

      • glueLakeFormationEnabled

        public boolean glueLakeFormationEnabled()

      • setGlueLakeFormationEnabled

        public void setGlueLakeFormationEnabled​(boolean glueLakeFormationEnabled)

      • dynamoDbTableName

        public java.lang.String dynamoDbTableName()

      • setDynamoDbTableName

        public void setDynamoDbTableName​(java.lang.String name)

      • applyGlueEndpointConfigurations

        public <T extends software.amazon.awssdk.services.glue.GlueClientBuilder> void applyGlueEndpointConfigurations​(T builder)
        Override the endpoint for a glue client.

        Sample usage: 

             GlueClient.builder().applyMutation(awsProperties::applyS3EndpointConfigurations)

      • applyDynamoDbEndpointConfigurations

        public <T extends software.amazon.awssdk.services.dynamodb.DynamoDbClientBuilder> void applyDynamoDbEndpointConfigurations​(T builder)
        Override the endpoint for a dynamoDb client.

        Sample usage: 

             DynamoDbClient.builder().applyMutation(awsProperties::applyDynamoDbEndpointConfigurations)

      • restSigningRegion

        public software.amazon.awssdk.regions.Region restSigningRegion()

      • restSigningName

        public java.lang.String restSigningName()

      • restCredentialsProvider

        public software.amazon.awssdk.auth.credentials.AwsCredentialsProvider restCredentialsProvider()