Package org.apache.iceberg.spark.source
Class SparkPartitionedFanoutWriter
- java.lang.Object
-
- org.apache.iceberg.io.BaseTaskWriter<T>
-
- org.apache.iceberg.io.PartitionedFanoutWriter<org.apache.spark.sql.catalyst.InternalRow>
-
- org.apache.iceberg.spark.source.SparkPartitionedFanoutWriter
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
,TaskWriter<org.apache.spark.sql.catalyst.InternalRow>
public class SparkPartitionedFanoutWriter extends PartitionedFanoutWriter<org.apache.spark.sql.catalyst.InternalRow>
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.iceberg.io.BaseTaskWriter
BaseTaskWriter.BaseEqualityDeltaWriter, BaseTaskWriter.RollingEqDeleteWriter, BaseTaskWriter.RollingFileWriter
-
-
Constructor Summary
Constructors Constructor Description SparkPartitionedFanoutWriter(PartitionSpec spec, FileFormat format, FileAppenderFactory<org.apache.spark.sql.catalyst.InternalRow> appenderFactory, OutputFileFactory fileFactory, FileIO io, long targetFileSize, Schema schema, org.apache.spark.sql.types.StructType sparkSchema)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected PartitionKey
partition(org.apache.spark.sql.catalyst.InternalRow row)
Create a PartitionKey from the values in row.-
Methods inherited from class org.apache.iceberg.io.PartitionedFanoutWriter
close, write
-
Methods inherited from class org.apache.iceberg.io.BaseTaskWriter
abort, complete, setFailure, spec
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.iceberg.io.TaskWriter
dataFiles
-
-
-
-
Constructor Detail
-
SparkPartitionedFanoutWriter
public SparkPartitionedFanoutWriter(PartitionSpec spec, FileFormat format, FileAppenderFactory<org.apache.spark.sql.catalyst.InternalRow> appenderFactory, OutputFileFactory fileFactory, FileIO io, long targetFileSize, Schema schema, org.apache.spark.sql.types.StructType sparkSchema)
-
-
Method Detail
-
partition
protected PartitionKey partition(org.apache.spark.sql.catalyst.InternalRow row)
Description copied from class:PartitionedFanoutWriter
Create a PartitionKey from the values in row.Any PartitionKey returned by this method can be reused by the implementation.
- Specified by:
partition
in classPartitionedFanoutWriter<org.apache.spark.sql.catalyst.InternalRow>
- Parameters:
row
- a data row
-
-