Package org.apache.iceberg.aws

Class AwsProperties

java.lang.Object
org.apache.iceberg.aws.AwsProperties
All Implemented Interfaces:
Serializable
public class AwsProperties extends Object implements Serializable
See Also:

  • Field Details

    • GLUE_CATALOG_ID

      public static final String GLUE_CATALOG_ID
      The ID of the Glue Data Catalog where the tables reside. If none is provided, Glue automatically uses the caller's AWS account ID by default.

      For more details, see https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-databases.html

      See Also:

    • GLUE_ACCOUNT_ID

      public static final String GLUE_ACCOUNT_ID
      The account ID used in a Glue resource ARN, e.g. arn:aws:glue:us-east-1:1000000000000:table/db1/table1
      See Also:

    • GLUE_CATALOG_SKIP_ARCHIVE

      public static final String GLUE_CATALOG_SKIP_ARCHIVE
      If Glue should skip archiving an old table version when creating a new version in a commit. By default Glue archives all old table versions after an UpdateTable call, but Glue has a default max number of archived table versions (can be increased). So for streaming use case with lots of commits, it is recommended to set this value to true.
      See Also:

    • GLUE_CATALOG_SKIP_ARCHIVE_DEFAULT

      public static final boolean GLUE_CATALOG_SKIP_ARCHIVE_DEFAULT
      See Also:

    • GLUE_CATALOG_SKIP_NAME_VALIDATION

      public static final String GLUE_CATALOG_SKIP_NAME_VALIDATION
      If Glue should skip name validations It is recommended to stick to Glue best practice in https://docs.aws.amazon.com/athena/latest/ug/glue-best-practices.html to make sure operations are Hive compatible. This is only added for users that have existing conventions using non-standard characters. When database name and table name validation are skipped, there is no guarantee that downstream systems would all support the names.
      See Also:

    • GLUE_CATALOG_SKIP_NAME_VALIDATION_DEFAULT

      public static final boolean GLUE_CATALOG_SKIP_NAME_VALIDATION_DEFAULT
      See Also:

    • GLUE_LAKEFORMATION_ENABLED

      public static final String GLUE_LAKEFORMATION_ENABLED
      If set, GlueCatalog will use Lake Formation for access control. For more credential vending details, see: https://docs.aws.amazon.com/lake-formation/latest/dg/api-overview.html. If enabled, the AwsClientFactory implementation must be LakeFormationAwsClientFactory or any class that extends it.
      See Also:

    • GLUE_LAKEFORMATION_ENABLED_DEFAULT

      public static final boolean GLUE_LAKEFORMATION_ENABLED_DEFAULT
      See Also:

    • GLUE_CATALOG_ENDPOINT

      public static final String GLUE_CATALOG_ENDPOINT
      Configure an alternative endpoint of the Glue service for GlueCatalog to access.

      This could be used to use GlueCatalog with any glue-compatible metastore service that has a different endpoint

      See Also:

    • DYNAMODB_ENDPOINT

      public static final String DYNAMODB_ENDPOINT
      Configure an alternative endpoint of the DynamoDB service to access.
      See Also:

    • DYNAMODB_TABLE_NAME

      public static final String DYNAMODB_TABLE_NAME
      DynamoDB table name for DynamoDbCatalog
      See Also:

    • DYNAMODB_TABLE_NAME_DEFAULT

      public static final String DYNAMODB_TABLE_NAME_DEFAULT
      See Also:

    • CLIENT_FACTORY

      public static final String CLIENT_FACTORY
      The implementation class of AwsClientFactory to customize AWS client configurations. If set, all AWS clients will be initialized by the specified factory. If not set, AwsClientFactories.defaultFactory() is used as default factory.
      See Also:

    • CLIENT_ASSUME_ROLE_ARN

      public static final String CLIENT_ASSUME_ROLE_ARN
      Used by AssumeRoleAwsClientFactory. If set, all AWS clients will assume a role of the given ARN, instead of using the default credential chain.
      See Also:

    • CLIENT_ASSUME_ROLE_TAGS_PREFIX

      public static final String CLIENT_ASSUME_ROLE_TAGS_PREFIX
      Used by AssumeRoleAwsClientFactory to pass a list of sessions. Each session tag consists of a key name and an associated value.
      See Also:

    • CLIENT_ASSUME_ROLE_TIMEOUT_SEC

      public static final String CLIENT_ASSUME_ROLE_TIMEOUT_SEC
      Used by AssumeRoleAwsClientFactory. The timeout of the assume role session in seconds, default to 1 hour. At the end of the timeout, a new set of role session credentials will be fetched through a STS client.
      See Also:

    • CLIENT_ASSUME_ROLE_TIMEOUT_SEC_DEFAULT

      public static final int CLIENT_ASSUME_ROLE_TIMEOUT_SEC_DEFAULT
      See Also:

    • CLIENT_ASSUME_ROLE_EXTERNAL_ID

      public static final String CLIENT_ASSUME_ROLE_EXTERNAL_ID
      Used by AssumeRoleAwsClientFactory. Optional external ID used to assume an IAM role.

      For more details, see https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html

      See Also:

    • CLIENT_ASSUME_ROLE_REGION

      public static final String CLIENT_ASSUME_ROLE_REGION
      Used by AssumeRoleAwsClientFactory. If set, all AWS clients except STS client will use the given region instead of the default region chain.

      The value must be one of Region, such as 'us-east-1'. For more details, see https://docs.aws.amazon.com/general/latest/gr/rande.html

      See Also:

    • CLIENT_ASSUME_ROLE_SESSION_NAME

      public static final String CLIENT_ASSUME_ROLE_SESSION_NAME
      Used by AssumeRoleAwsClientFactory. Optional session name used to assume an IAM role.

      For more details, see https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_iam-condition-keys.html#ck_rolesessionname

      See Also:

    • LAKE_FORMATION_TABLE_NAME

      public static final String LAKE_FORMATION_TABLE_NAME
      Used by LakeFormationAwsClientFactory. The table name used as part of lake formation credentials request.
      See Also:

    • LAKE_FORMATION_DB_NAME

      public static final String LAKE_FORMATION_DB_NAME
      Used by LakeFormationAwsClientFactory. The database name used as part of lake formation credentials request.
      See Also:

    • REST_SIGNER_REGION

      public static final String REST_SIGNER_REGION
      Region to be used by the SigV4 protocol for signing requests.
      See Also:

    • REST_SIGNING_NAME

      public static final String REST_SIGNING_NAME
      The service name to be used by the SigV4 protocol for signing requests.
      See Also:

    • REST_SIGNING_NAME_DEFAULT

      public static final String REST_SIGNING_NAME_DEFAULT
      The default service name (API Gateway and lambda) used during SigV4 signing.
      See Also:

    • REST_ACCESS_KEY_ID

      public static final String REST_ACCESS_KEY_ID
      Configure the static access key ID used for SigV4 signing.

      When set, the default client factory will use the basic or session credentials provided instead of reading the default credential chain to create S3 access credentials. If REST_SESSION_TOKEN is set, session credential is used, otherwise basic credential is used.

      See Also:

    • REST_SECRET_ACCESS_KEY

      public static final String REST_SECRET_ACCESS_KEY
      Configure the static secret access key used for SigV4 signing.

      When set, the default client factory will use the basic or session credentials provided instead of reading the default credential chain to create S3 access credentials. If REST_SESSION_TOKEN is set, session credential is used, otherwise basic credential is used.

      See Also:

    • REST_SESSION_TOKEN

      public static final String REST_SESSION_TOKEN
      Configure the static session token used for SigV4.

      When set, the default client factory will use the session credentials provided instead of reading the default credential chain to create access credentials.

      See Also:

    • KMS_ENCRYPTION_ALGORITHM_SPEC

      public static final String KMS_ENCRYPTION_ALGORITHM_SPEC
      Encryption algorithm used to encrypt/decrypt master table keys
      See Also:

    • KMS_ENCRYPTION_ALGORITHM_SPEC_DEFAULT

      public static final software.amazon.awssdk.services.kms.model.EncryptionAlgorithmSpec KMS_ENCRYPTION_ALGORITHM_SPEC_DEFAULT

    • KMS_DATA_KEY_SPEC

      public static final String KMS_DATA_KEY_SPEC
      Length of data key generated by KMS
      See Also:

    • KMS_DATA_KEY_SPEC_DEFAULT

      public static final software.amazon.awssdk.services.kms.model.DataKeySpec KMS_DATA_KEY_SPEC_DEFAULT

  • Constructor Details

    • AwsProperties

      public AwsProperties()

    • AwsProperties

      public AwsProperties(Map<String,String> properties)

  • Method Details

    • stsClientAssumeRoleTags

      public Set<software.amazon.awssdk.services.sts.model.Tag> stsClientAssumeRoleTags()

    • clientAssumeRoleArn

      public String clientAssumeRoleArn()

    • clientAssumeRoleTimeoutSec

      public int clientAssumeRoleTimeoutSec()

    • clientAssumeRoleExternalId

      public String clientAssumeRoleExternalId()

    • clientAssumeRoleRegion

      public String clientAssumeRoleRegion()

    • clientAssumeRoleSessionName

      public String clientAssumeRoleSessionName()

    • glueCatalogId

      public String glueCatalogId()

    • setGlueCatalogId

      public void setGlueCatalogId(String id)

    • glueCatalogSkipArchive

      public boolean glueCatalogSkipArchive()

    • setGlueCatalogSkipArchive

      public void setGlueCatalogSkipArchive(boolean skipArchive)

    • glueCatalogSkipNameValidation

      public boolean glueCatalogSkipNameValidation()

    • setGlueCatalogSkipNameValidation

      public void setGlueCatalogSkipNameValidation(boolean glueCatalogSkipNameValidation)

    • glueLakeFormationEnabled

      public boolean glueLakeFormationEnabled()

    • setGlueLakeFormationEnabled

      public void setGlueLakeFormationEnabled(boolean glueLakeFormationEnabled)

    • dynamoDbTableName

      public String dynamoDbTableName()

    • setDynamoDbTableName

      public void setDynamoDbTableName(String name)

    • applyGlueEndpointConfigurations

      public <T extends software.amazon.awssdk.services.glue.GlueClientBuilder> void applyGlueEndpointConfigurations(T builder)
      Override the endpoint for a glue client.

      Sample usage: 

           GlueClient.builder().applyMutation(awsProperties::applyS3EndpointConfigurations)

    • applyDynamoDbEndpointConfigurations

      public <T extends software.amazon.awssdk.services.dynamodb.DynamoDbClientBuilder> void applyDynamoDbEndpointConfigurations(T builder)
      Override the endpoint for a dynamoDb client.

      Sample usage: 

           DynamoDbClient.builder().applyMutation(awsProperties::applyDynamoDbEndpointConfigurations)

    • restSigningRegion

      public software.amazon.awssdk.regions.Region restSigningRegion()

    • restSigningName

      public String restSigningName()

    • restCredentialsProvider

      public software.amazon.awssdk.auth.credentials.AwsCredentialsProvider restCredentialsProvider()

    • kmsEncryptionAlgorithmSpec

      public software.amazon.awssdk.services.kms.model.EncryptionAlgorithmSpec kmsEncryptionAlgorithmSpec()

    • kmsDataKeySpec

      public software.amazon.awssdk.services.kms.model.DataKeySpec kmsDataKeySpec()