Class SparkWriteUtil


  • public class SparkWriteUtil
    extends java.lang.Object
    A utility that contains helper methods for working with Spark writes.

    Note it is an evolving internal API that is subject to change even in minor releases.

    • Method Detail

      • writeRequirements

        public static SparkWriteRequirements writeRequirements​(Table table,
                                                               DistributionMode mode,
                                                               boolean fanoutEnabled,
                                                               long advisoryPartitionSize)
        Builds requirements for batch and micro-batch writes such as append or overwrite.
      • copyOnWriteRequirements

        public static SparkWriteRequirements copyOnWriteRequirements​(Table table,
                                                                     org.apache.spark.sql.connector.write.RowLevelOperation.Command command,
                                                                     DistributionMode mode,
                                                                     boolean fanoutEnabled,
                                                                     long advisoryPartitionSize)
        Builds requirements for copy-on-write DELETE, UPDATE, MERGE operations.
      • positionDeltaRequirements

        public static SparkWriteRequirements positionDeltaRequirements​(Table table,
                                                                       org.apache.spark.sql.connector.write.RowLevelOperation.Command command,
                                                                       DistributionMode mode,
                                                                       boolean fanoutEnabled,
                                                                       long advisoryPartitionSize)
        Builds requirements for merge-on-read DELETE, UPDATE, MERGE operations.