Package org.apache.iceberg.io

Class RollingDataWriter<T>

java.lang.Object

org.apache.iceberg.io.RollingDataWriter<T>

All Implemented Interfaces:

Closeable, AutoCloseable, FileWriter<T,DataWriteResult>

public class RollingDataWriter<T>
extends Object

A rolling data writer that splits incoming data into multiple files within one spec/partition based on the target file size.

Constructor Summary

Constructors

Constructor	Description
RollingDataWriter(FileWriterFactory<T> writerFactory, OutputFileFactory fileFactory, FileIO io, long targetFileSizeInBytes, PartitionSpec spec, StructLike partition)

Method Summary

Modifier and Type	Method	Description
protected void	addResult(DataWriteResult result)
protected DataWriteResult	aggregatedResult()
void	close()
CharSequence	currentFilePath()
long	currentFileRows()
long	length()	Returns the number of bytes that were currently written by this writer.
protected DataWriter<T>	newWriter(EncryptedOutputFile file)
protected void	openCurrentWriter()
protected StructLike	partition()
final DataWriteResult	result()	Returns a result that contains information about written DataFiles or DeleteFiles.
protected PartitionSpec	spec()
void	write(T row)	Writes a row to a predefined spec/partition.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.io.FileWriter

write

Constructor Details

RollingDataWriter

public RollingDataWriter(FileWriterFactory<T> writerFactory,
 OutputFileFactory fileFactory,
 FileIO io,
 long targetFileSizeInBytes,
 PartitionSpec spec,
 StructLike partition)

Method Details

newWriter

protected DataWriter<T> newWriter(EncryptedOutputFile file)

addResult

protected void addResult(DataWriteResult result)

aggregatedResult

protected DataWriteResult aggregatedResult()

spec

protected PartitionSpec spec()

partition

protected StructLike partition()

currentFilePath

public CharSequence currentFilePath()

currentFileRows

public long currentFileRows()

length

public long length()

Description copied from interface: FileWriter

Returns the number of bytes that were currently written by this writer.

Specified by:

length in interface FileWriter<T,W extends FileWriter<T,R>>

Returns:

the number of written bytes

write

public void write(T row)

Description copied from interface: FileWriter

Writes a row to a predefined spec/partition.

Specified by:

write in interface FileWriter<T,W extends FileWriter<T,R>>

Parameters:

row - a data or delete record

openCurrentWriter

protected void openCurrentWriter()

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

result

public final DataWriteResult result()

Description copied from interface: FileWriter

Returns a result that contains information about written DataFiles or DeleteFiles. The result is valid only after the writer is closed.

Specified by:

result in interface FileWriter<T,W extends FileWriter<T,R>>

Returns:

the file writer result