public class VectorizedArrowReader extends java.lang.Object implements VectorizedReader<VectorHolder>
VectorReader(s)
that read in a batch of values into Arrow vectors.
It also takes care of allocating the right kind of Arrow vectors depending on the corresponding
Iceberg/Parquet data types.Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_BATCH_SIZE |
static VectorizedArrowReader |
NULL_VALUES_READER |
Constructor and Description |
---|
VectorizedArrowReader(org.apache.parquet.column.ColumnDescriptor desc,
Types.NestedField icebergField,
org.apache.arrow.memory.BufferAllocator ra,
int batchSize,
boolean setArrowValidityVector) |
Modifier and Type | Method and Description |
---|---|
void |
close()
Release any resources allocated
|
VectorHolder |
read(int numValsToRead)
Reads a batch of type @param <T> and of size numRows
|
void |
reuseContainers(boolean reuse)
Set up the reader to reuse the underlying containers used for storing batches
|
void |
setRowGroupInfo(org.apache.parquet.column.page.PageReadStore source,
java.util.Map<org.apache.parquet.hadoop.metadata.ColumnPath,org.apache.parquet.hadoop.metadata.ColumnChunkMetaData> metadata) |
java.lang.String |
toString() |
public static final int DEFAULT_BATCH_SIZE
public static final VectorizedArrowReader NULL_VALUES_READER
public VectorizedArrowReader(org.apache.parquet.column.ColumnDescriptor desc, Types.NestedField icebergField, org.apache.arrow.memory.BufferAllocator ra, int batchSize, boolean setArrowValidityVector)
public VectorHolder read(int numValsToRead)
VectorizedReader
read
in interface VectorizedReader<VectorHolder>
numValsToRead
- number of rows to readpublic void setRowGroupInfo(org.apache.parquet.column.page.PageReadStore source, java.util.Map<org.apache.parquet.hadoop.metadata.ColumnPath,org.apache.parquet.hadoop.metadata.ColumnChunkMetaData> metadata)
setRowGroupInfo
in interface VectorizedReader<VectorHolder>
source
- row group information for all the columnsmetadata
- map of ColumnPath
-> ColumnChunkMetaData
for the row grouppublic void reuseContainers(boolean reuse)
VectorizedReader
reuseContainers
in interface VectorizedReader<VectorHolder>
public void close()
VectorizedReader
close
in interface VectorizedReader<VectorHolder>
public java.lang.String toString()
toString
in class java.lang.Object