Package org.apache.iceberg.spark
Class SparkWriteUtil
- java.lang.Object
-
- org.apache.iceberg.spark.SparkWriteUtil
-
public class SparkWriteUtil extends java.lang.Object
A utility that contains helper methods for working with Spark writes.Note it is an evolving internal API that is subject to change even in minor releases.
-
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static SparkWriteRequirements
copyOnWriteRequirements(Table table, org.apache.spark.sql.connector.write.RowLevelOperation.Command command, DistributionMode mode, boolean fanoutEnabled, long advisoryPartitionSize)
Builds requirements for copy-on-write DELETE, UPDATE, MERGE operations.static SparkWriteRequirements
positionDeltaRequirements(Table table, org.apache.spark.sql.connector.write.RowLevelOperation.Command command, DistributionMode mode, boolean fanoutEnabled, long advisoryPartitionSize)
Builds requirements for merge-on-read DELETE, UPDATE, MERGE operations.static SparkWriteRequirements
writeRequirements(Table table, DistributionMode mode, boolean fanoutEnabled, long advisoryPartitionSize)
Builds requirements for batch and micro-batch writes such as append or overwrite.
-
-
-
Method Detail
-
writeRequirements
public static SparkWriteRequirements writeRequirements(Table table, DistributionMode mode, boolean fanoutEnabled, long advisoryPartitionSize)
Builds requirements for batch and micro-batch writes such as append or overwrite.
-
copyOnWriteRequirements
public static SparkWriteRequirements copyOnWriteRequirements(Table table, org.apache.spark.sql.connector.write.RowLevelOperation.Command command, DistributionMode mode, boolean fanoutEnabled, long advisoryPartitionSize)
Builds requirements for copy-on-write DELETE, UPDATE, MERGE operations.
-
positionDeltaRequirements
public static SparkWriteRequirements positionDeltaRequirements(Table table, org.apache.spark.sql.connector.write.RowLevelOperation.Command command, DistributionMode mode, boolean fanoutEnabled, long advisoryPartitionSize)
Builds requirements for merge-on-read DELETE, UPDATE, MERGE operations.
-
-