Package org.apache.iceberg.spark.source
Class SparkPartitionedWriter
- java.lang.Object
-
- org.apache.iceberg.io.BaseTaskWriter<T>
-
- org.apache.iceberg.io.PartitionedWriter<org.apache.spark.sql.catalyst.InternalRow>
-
- org.apache.iceberg.spark.source.SparkPartitionedWriter
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
,TaskWriter<org.apache.spark.sql.catalyst.InternalRow>
public class SparkPartitionedWriter extends PartitionedWriter<org.apache.spark.sql.catalyst.InternalRow>
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.iceberg.io.BaseTaskWriter
BaseTaskWriter.BaseEqualityDeltaWriter, BaseTaskWriter.RollingEqDeleteWriter, BaseTaskWriter.RollingFileWriter
-
-
Constructor Summary
Constructors Constructor Description SparkPartitionedWriter(PartitionSpec spec, FileFormat format, FileAppenderFactory<org.apache.spark.sql.catalyst.InternalRow> appenderFactory, OutputFileFactory fileFactory, FileIO io, long targetFileSize, Schema schema, org.apache.spark.sql.types.StructType sparkSchema)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected PartitionKey
partition(org.apache.spark.sql.catalyst.InternalRow row)
Create a PartitionKey from the values in row.-
Methods inherited from class org.apache.iceberg.io.PartitionedWriter
close, write
-
Methods inherited from class org.apache.iceberg.io.BaseTaskWriter
abort, complete, setFailure, spec
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.iceberg.io.TaskWriter
dataFiles
-
-
-
-
Constructor Detail
-
SparkPartitionedWriter
public SparkPartitionedWriter(PartitionSpec spec, FileFormat format, FileAppenderFactory<org.apache.spark.sql.catalyst.InternalRow> appenderFactory, OutputFileFactory fileFactory, FileIO io, long targetFileSize, Schema schema, org.apache.spark.sql.types.StructType sparkSchema)
-
-
Method Detail
-
partition
protected PartitionKey partition(org.apache.spark.sql.catalyst.InternalRow row)
Description copied from class:PartitionedWriter
Create a PartitionKey from the values in row.Any PartitionKey returned by this method can be reused by the implementation.
- Specified by:
partition
in classPartitionedWriter<org.apache.spark.sql.catalyst.InternalRow>
- Parameters:
row
- a data row
-
-