Package org.apache.iceberg.io
Class FanoutDataWriter<T>
- java.lang.Object
-
- org.apache.iceberg.io.FanoutDataWriter<T>
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
,PartitioningWriter<T,DataWriteResult>
public class FanoutDataWriter<T> extends java.lang.Object
A data writer capable of writing to multiple specs and partitions that keeps data writers for each seen spec/partition pair open until this writer is closed.
-
-
Constructor Summary
Constructors Constructor Description FanoutDataWriter(FileWriterFactory<T> writerFactory, OutputFileFactory fileFactory, FileIO io, FileFormat fileFormat, long targetFileSizeInBytes)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected void
addResult(DataWriteResult result)
protected DataWriteResult
aggregatedResult()
void
close()
protected EncryptedOutputFile
newOutputFile(OutputFileFactory fileFactory, PartitionSpec spec, StructLike partition)
protected FileWriter<T,DataWriteResult>
newWriter(PartitionSpec spec, StructLike partition)
R
result()
Returns a result that contains information about writtenDataFile
s orDeleteFile
s.void
write(T row, PartitionSpec spec, StructLike partition)
Writes a row to the provided spec/partition.
-
-
-
Constructor Detail
-
FanoutDataWriter
public FanoutDataWriter(FileWriterFactory<T> writerFactory, OutputFileFactory fileFactory, FileIO io, FileFormat fileFormat, long targetFileSizeInBytes)
-
-
Method Detail
-
newWriter
protected FileWriter<T,DataWriteResult> newWriter(PartitionSpec spec, StructLike partition)
-
addResult
protected void addResult(DataWriteResult result)
-
aggregatedResult
protected DataWriteResult aggregatedResult()
-
write
public void write(T row, PartitionSpec spec, StructLike partition)
Description copied from interface:PartitioningWriter
Writes a row to the provided spec/partition.- Specified by:
write
in interfacePartitioningWriter<T,R>
- Parameters:
row
- a data or delete recordspec
- a partition specpartition
- a partition or null if the spec is unpartitioned
-
close
public void close() throws java.io.IOException
- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfacejava.io.Closeable
- Throws:
java.io.IOException
-
result
public final R result()
Description copied from interface:PartitioningWriter
Returns a result that contains information about writtenDataFile
s orDeleteFile
s. The result is valid only after the writer is closed.- Specified by:
result
in interfacePartitioningWriter<T,R>
- Returns:
- the writer result
-
newOutputFile
protected EncryptedOutputFile newOutputFile(OutputFileFactory fileFactory, PartitionSpec spec, StructLike partition)
-
-