Package org.apache.iceberg.spark.source

Class SparkTable

java.lang.Object
org.apache.iceberg.spark.source.SparkTable
All Implemented Interfaces:
org.apache.spark.sql.connector.catalog.SupportsDeleteV2, org.apache.spark.sql.connector.catalog.SupportsMetadataColumns, org.apache.spark.sql.connector.catalog.SupportsRead, org.apache.spark.sql.connector.catalog.SupportsRowLevelOperations, org.apache.spark.sql.connector.catalog.SupportsWrite, org.apache.spark.sql.connector.catalog.Table, org.apache.spark.sql.connector.catalog.TruncatableTable
Direct Known Subclasses:
StagedSparkTable
public class SparkTable extends Object implements org.apache.spark.sql.connector.catalog.SupportsRead, org.apache.spark.sql.connector.catalog.SupportsWrite, org.apache.spark.sql.connector.catalog.SupportsDeleteV2, org.apache.spark.sql.connector.catalog.SupportsRowLevelOperations
The main Spark table implementation that supports reads, writes, and row-level operations.

Note the table state (e.g. schema, snapshot) is pinned upon loading and must not change.

  • Constructor Details

    • SparkTable

      public SparkTable(Table table)

  • Method Details

    • copyWithBranch

      public SparkTable copyWithBranch(String newBranch)

    • snapshotId

      public Long snapshotId()

    • branch

      public String branch()

    • id

      public String id()
      Specified by:
      id in interface org.apache.spark.sql.connector.catalog.Table

    • version

      public String version()
      Specified by:
      version in interface org.apache.spark.sql.connector.catalog.Table

    • capabilities

      public Set<org.apache.spark.sql.connector.catalog.TableCapability> capabilities()
      Specified by:
      capabilities in interface org.apache.spark.sql.connector.catalog.Table

    • constraints

      public org.apache.spark.sql.connector.catalog.constraints.Constraint[] constraints()
      Specified by:
      constraints in interface org.apache.spark.sql.connector.catalog.Table

    • newScanBuilder

      public org.apache.spark.sql.connector.read.ScanBuilder newScanBuilder(org.apache.spark.sql.util.CaseInsensitiveStringMap options)
      Specified by:
      newScanBuilder in interface org.apache.spark.sql.connector.catalog.SupportsRead

    • newWriteBuilder

      public org.apache.spark.sql.connector.write.WriteBuilder newWriteBuilder(org.apache.spark.sql.connector.write.LogicalWriteInfo info)
      Specified by:
      newWriteBuilder in interface org.apache.spark.sql.connector.catalog.SupportsWrite

    • newRowLevelOperationBuilder

      public org.apache.spark.sql.connector.write.RowLevelOperationBuilder newRowLevelOperationBuilder(org.apache.spark.sql.connector.write.RowLevelOperationInfo info)
      Specified by:
      newRowLevelOperationBuilder in interface org.apache.spark.sql.connector.catalog.SupportsRowLevelOperations

    • canDeleteWhere

      public boolean canDeleteWhere(org.apache.spark.sql.connector.expressions.filter.Predicate[] predicates)
      Specified by:
      canDeleteWhere in interface org.apache.spark.sql.connector.catalog.SupportsDeleteV2

    • deleteWhere

      public void deleteWhere(org.apache.spark.sql.connector.expressions.filter.Predicate[] predicates)
      Specified by:
      deleteWhere in interface org.apache.spark.sql.connector.catalog.SupportsDeleteV2

    • equals

      public boolean equals(Object other)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • create

      public static SparkTable create(Table table, String branch)

    • create

      public static SparkTable create(Table table, TimeTravel timeTravel)

    • spark

      protected org.apache.spark.sql.SparkSession spark()

    • table

      public Table table()

    • name

      public String name()
      Specified by:
      name in interface org.apache.spark.sql.connector.catalog.Table

    • schema

      public org.apache.spark.sql.types.StructType schema()
      Specified by:
      schema in interface org.apache.spark.sql.connector.catalog.Table

    • partitioning

      public org.apache.spark.sql.connector.expressions.Transform[] partitioning()
      Specified by:
      partitioning in interface org.apache.spark.sql.connector.catalog.Table

    • properties

      public Map<String,String> properties()
      Specified by:
      properties in interface org.apache.spark.sql.connector.catalog.Table

    • metadataColumns

      public org.apache.spark.sql.connector.catalog.MetadataColumn[] metadataColumns()
      Specified by:
      metadataColumns in interface org.apache.spark.sql.connector.catalog.SupportsMetadataColumns

    • toString

      public String toString()
      Overrides:
      toString in class Object