Package org.apache.iceberg.spark.source
Class SparkPartitionedFanoutWriter
java.lang.Object
org.apache.iceberg.io.BaseTaskWriter<T>
org.apache.iceberg.io.PartitionedFanoutWriter<org.apache.spark.sql.catalyst.InternalRow>
org.apache.iceberg.spark.source.SparkPartitionedFanoutWriter
- All Implemented Interfaces:
Closeable
,AutoCloseable
,TaskWriter<org.apache.spark.sql.catalyst.InternalRow>
public class SparkPartitionedFanoutWriter
extends PartitionedFanoutWriter<org.apache.spark.sql.catalyst.InternalRow>
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.iceberg.io.BaseTaskWriter
BaseTaskWriter.BaseEqualityDeltaWriter, BaseTaskWriter.RollingEqDeleteWriter, BaseTaskWriter.RollingFileWriter
-
Constructor Summary
ConstructorDescriptionSparkPartitionedFanoutWriter
(PartitionSpec spec, FileFormat format, FileAppenderFactory<org.apache.spark.sql.catalyst.InternalRow> appenderFactory, OutputFileFactory fileFactory, FileIO io, long targetFileSize, Schema schema, org.apache.spark.sql.types.StructType sparkSchema) -
Method Summary
Modifier and TypeMethodDescriptionprotected PartitionKey
partition
(org.apache.spark.sql.catalyst.InternalRow row) Create a PartitionKey from the values in row.Methods inherited from class org.apache.iceberg.io.PartitionedFanoutWriter
close, write
Methods inherited from class org.apache.iceberg.io.BaseTaskWriter
abort, complete, setFailure, spec
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.iceberg.io.TaskWriter
dataFiles
-
Constructor Details
-
SparkPartitionedFanoutWriter
public SparkPartitionedFanoutWriter(PartitionSpec spec, FileFormat format, FileAppenderFactory<org.apache.spark.sql.catalyst.InternalRow> appenderFactory, OutputFileFactory fileFactory, FileIO io, long targetFileSize, Schema schema, org.apache.spark.sql.types.StructType sparkSchema)
-
-
Method Details
-
partition
Description copied from class:PartitionedFanoutWriter
Create a PartitionKey from the values in row.Any PartitionKey returned by this method can be reused by the implementation.
- Specified by:
partition
in classPartitionedFanoutWriter<org.apache.spark.sql.catalyst.InternalRow>
- Parameters:
row
- a data row
-