org.apache.iceberg.arrow.vectorized

Class VectorizedArrowReader

java.lang.Object

org.apache.iceberg.arrow.vectorized.VectorizedArrowReader

All Implemented Interfaces:

VectorizedReader<VectorHolder>

public class VectorizedArrowReader
extends java.lang.Object
implements VectorizedReader<VectorHolder>

VectorReader(s) that read in a batch of values into Arrow vectors. It also takes care of allocating the right kind of Arrow vectors depending on the corresponding Iceberg/Parquet data types.

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_BATCH_SIZE
static VectorizedArrowReader	NULL_VALUES_READER

Constructor Summary

Constructors

Constructor and Description
VectorizedArrowReader(org.apache.parquet.column.ColumnDescriptor desc, Types.NestedField icebergField, org.apache.arrow.memory.BufferAllocator ra, int batchSize, boolean setArrowValidityVector)

Method Summary

Modifier and Type	Method and Description
void	close() Release any resources allocated
VectorHolder	read(int numValsToRead) Reads a batch of type @param <T> and of size numRows
void	reuseContainers(boolean reuse) Set up the reader to reuse the underlying containers used for storing batches
void	setRowGroupInfo(org.apache.parquet.column.page.PageReadStore source, java.util.Map<org.apache.parquet.hadoop.metadata.ColumnPath,org.apache.parquet.hadoop.metadata.ColumnChunkMetaData> metadata)
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_BATCH_SIZE

public static final int DEFAULT_BATCH_SIZE

See Also:

Constant Field Values

NULL_VALUES_READER

public static final VectorizedArrowReader NULL_VALUES_READER

Constructor Detail

VectorizedArrowReader

public VectorizedArrowReader(org.apache.parquet.column.ColumnDescriptor desc,
                             Types.NestedField icebergField,
                             org.apache.arrow.memory.BufferAllocator ra,
                             int batchSize,
                             boolean setArrowValidityVector)

Method Detail

read

public VectorHolder read(int numValsToRead)

Description copied from interface: VectorizedReader

Reads a batch of type @param <T> and of size numRows

Specified by:

read in interface VectorizedReader<VectorHolder>

Parameters:

numValsToRead - number of rows to read

Returns:

batch of records of type @param <T>

setRowGroupInfo

public void setRowGroupInfo(org.apache.parquet.column.page.PageReadStore source,
                            java.util.Map<org.apache.parquet.hadoop.metadata.ColumnPath,org.apache.parquet.hadoop.metadata.ColumnChunkMetaData> metadata)

Specified by:

setRowGroupInfo in interface VectorizedReader<VectorHolder>

Parameters:

source - row group information for all the columns

metadata - map of ColumnPath -> ColumnChunkMetaData for the row group

reuseContainers

public void reuseContainers(boolean reuse)

Description copied from interface: VectorizedReader

Set up the reader to reuse the underlying containers used for storing batches

Specified by:

reuseContainers in interface VectorizedReader<VectorHolder>

close

public void close()

Description copied from interface: VectorizedReader

Release any resources allocated

Specified by:

close in interface VectorizedReader<VectorHolder>

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object