org.apache.iceberg.actions

Class Spark3SnapshotAction

java.lang.Object

org.apache.iceberg.actions.Spark3SnapshotAction

All Implemented Interfaces:

Action<java.lang.Long>, CreateAction, SnapshotAction

public class Spark3SnapshotAction
extends java.lang.Object
implements SnapshotAction

Creates a new Iceberg table based on a source Spark table. The new Iceberg table will have a different data and metadata directory allowing it to exist independently of the source table.

Field Summary

Fields

Modifier and Type	Field and Description
protected static java.util.List<java.lang.String>	EXCLUDED_PROPERTIES
protected static java.lang.String	ICEBERG_METADATA_FOLDER
protected static java.lang.String	LOCATION

Constructor Summary

Constructors

Constructor and Description
Spark3SnapshotAction(org.apache.spark.sql.SparkSession spark, org.apache.spark.sql.connector.catalog.CatalogPlugin sourceCatalog, org.apache.spark.sql.connector.catalog.Identifier sourceTableIdent, org.apache.spark.sql.connector.catalog.CatalogPlugin destCatalog, org.apache.spark.sql.connector.catalog.Identifier destTableIdent)

Method Summary

Modifier and Type	Method and Description
protected java.util.Map<java.lang.String,java.lang.String>	additionalProperties()
protected org.apache.spark.sql.connector.catalog.TableCatalog	checkSourceCatalog(org.apache.spark.sql.connector.catalog.CatalogPlugin catalog)
protected org.apache.spark.sql.connector.catalog.StagingTableCatalog	destCatalog()
protected org.apache.spark.sql.connector.catalog.Identifier	destTableIdent()
protected void	ensureNameMappingPresent(Table table)
java.lang.Long	execute() Executes this action.
protected java.lang.String	getMetadataLocation(Table table)
protected org.apache.spark.sql.connector.catalog.TableCatalog	sourceCatalog()
protected org.apache.spark.sql.connector.catalog.Identifier	sourceTableIdent()
protected java.lang.String	sourceTableLocation()
protected org.apache.spark.sql.SparkSession	spark()
protected StagedSparkTable	stageDestTable()
protected java.util.Map<java.lang.String,java.lang.String>	targetTableProps()
protected org.apache.spark.sql.catalyst.catalog.CatalogTable	v1SourceTable()
SnapshotAction	withLocation(java.lang.String location)
CreateAction	withProperties(java.util.Map<java.lang.String,java.lang.String> properties) Adds additional properties to the newly created Iceberg table.
CreateAction	withProperty(java.lang.String key, java.lang.String value) Adds an additional property to the newly created Iceberg table.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.actions.CreateAction

withProperties, withProperty

Field Detail

LOCATION

protected static final java.lang.String LOCATION

See Also:

Constant Field Values

ICEBERG_METADATA_FOLDER

protected static final java.lang.String ICEBERG_METADATA_FOLDER

See Also:

Constant Field Values

EXCLUDED_PROPERTIES

protected static final java.util.List<java.lang.String> EXCLUDED_PROPERTIES

Constructor Detail

Spark3SnapshotAction

public Spark3SnapshotAction(org.apache.spark.sql.SparkSession spark,
                            org.apache.spark.sql.connector.catalog.CatalogPlugin sourceCatalog,
                            org.apache.spark.sql.connector.catalog.Identifier sourceTableIdent,
                            org.apache.spark.sql.connector.catalog.CatalogPlugin destCatalog,
                            org.apache.spark.sql.connector.catalog.Identifier destTableIdent)

Method Detail

execute

public java.lang.Long execute()

Description copied from interface: Action

Executes this action.

Specified by:

execute in interface Action<java.lang.Long>

Returns:

the result of this action

targetTableProps

protected java.util.Map<java.lang.String,java.lang.String> targetTableProps()

checkSourceCatalog

protected org.apache.spark.sql.connector.catalog.TableCatalog checkSourceCatalog(org.apache.spark.sql.connector.catalog.CatalogPlugin catalog)

withLocation

public SnapshotAction withLocation(java.lang.String location)

Specified by:

withLocation in interface SnapshotAction

withProperties

public CreateAction withProperties(java.util.Map<java.lang.String,java.lang.String> properties)

Description copied from interface: CreateAction

Adds additional properties to the newly created Iceberg table. Any properties with the same key name will be overwritten.

Specified by:

withProperties in interface CreateAction

Parameters:

properties - a map of properties to be included

Returns:

this for chaining

withProperty

public CreateAction withProperty(java.lang.String key,
                                 java.lang.String value)

Description copied from interface: CreateAction

Adds an additional property to the newly created Iceberg table. Any properties with the same key name will be overwritten.

Specified by:

withProperty in interface CreateAction

Parameters:

key - the key of the property to add

value - the value of the property to add

Returns:

this for chaining

spark

protected org.apache.spark.sql.SparkSession spark()

sourceTableLocation

protected java.lang.String sourceTableLocation()

v1SourceTable

protected org.apache.spark.sql.catalyst.catalog.CatalogTable v1SourceTable()

sourceCatalog

protected org.apache.spark.sql.connector.catalog.TableCatalog sourceCatalog()

sourceTableIdent

protected org.apache.spark.sql.connector.catalog.Identifier sourceTableIdent()

destCatalog

protected org.apache.spark.sql.connector.catalog.StagingTableCatalog destCatalog()

destTableIdent

protected org.apache.spark.sql.connector.catalog.Identifier destTableIdent()

additionalProperties

protected java.util.Map<java.lang.String,java.lang.String> additionalProperties()

stageDestTable

protected StagedSparkTable stageDestTable()

ensureNameMappingPresent

protected void ensureNameMappingPresent(Table table)

getMetadataLocation

protected java.lang.String getMetadataLocation(Table table)