Package org.apache.iceberg

Class PartitionStatsHandler

java.lang.Object

org.apache.iceberg.PartitionStatsHandler

public class PartitionStatsHandler
extends Object

Computes, writes and reads the PartitionStatisticsFile. Uses generic readers and writers to support writing and reading of the stats in table default format.

Field Summary

Fields

Modifier and Type	Field	Description
static final Types.NestedField	DATA_FILE_COUNT	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	DATA_RECORD_COUNT	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	DV_COUNT	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	EQUALITY_DELETE_FILE_COUNT	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	EQUALITY_DELETE_RECORD_COUNT	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	LAST_UPDATED_AT	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	LAST_UPDATED_SNAPSHOT_ID	Deprecated. will be removed in 1.12.0.
static final int	PARTITION_FIELD_ID	Deprecated. will be removed in 1.12.0.
static final String	PARTITION_FIELD_NAME	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	POSITION_DELETE_FILE_COUNT	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	POSITION_DELETE_RECORD_COUNT	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	SPEC_ID	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	TOTAL_DATA_FILE_SIZE_IN_BYTES	Deprecated. will be removed in 1.12.0.
static final Types.NestedField	TOTAL_RECORD_COUNT	Deprecated. will be removed in 1.12.0.

Method Summary

Modifier and Type	Method	Description
static PartitionStatisticsFile	computeAndWriteStatsFile(Table table)	Computes the stats incrementally after the snapshot that has partition stats file till the current snapshot and writes the combined result into a PartitionStatisticsFile after merging the stats for a given table's current snapshot.
static PartitionStatisticsFile	computeAndWriteStatsFile(Table table, long snapshotId)	Computes the stats incrementally after the snapshot that has partition stats file till the given snapshot and writes the combined result into a PartitionStatisticsFile after merging the stats for a given snapshot.
static CloseableIterable<PartitionStats>	readPartitionStatsFile(Schema schema, InputFile inputFile)	Deprecated. will be removed in 1.12.0, use PartitionStatisticsScan instead
static Schema	schema(Types.StructType unifiedPartitionType, int formatVersion)	Deprecated. will be removed in 1.12.0.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

PARTITION_FIELD_ID

@Deprecated
public static final int PARTITION_FIELD_ID

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.EMPTY_PARTITION_FIELD

See Also:

• Constant Field Values

PARTITION_FIELD_NAME

@Deprecated
public static final String PARTITION_FIELD_NAME

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.EMPTY_PARTITION_FIELD

See Also:

• Constant Field Values

SPEC_ID

@Deprecated
public static final Types.NestedField SPEC_ID

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.SPEC_ID

DATA_RECORD_COUNT

@Deprecated
public static final Types.NestedField DATA_RECORD_COUNT

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.DATA_RECORD_COUNT

DATA_FILE_COUNT

@Deprecated
public static final Types.NestedField DATA_FILE_COUNT

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.DATA_FILE_COUNT

TOTAL_DATA_FILE_SIZE_IN_BYTES

@Deprecated
public static final Types.NestedField TOTAL_DATA_FILE_SIZE_IN_BYTES

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.TOTAL_DATA_FILE_SIZE_IN_BYTES

POSITION_DELETE_RECORD_COUNT

@Deprecated
public static final Types.NestedField POSITION_DELETE_RECORD_COUNT

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.POSITION_DELETE_RECORD_COUNT

POSITION_DELETE_FILE_COUNT

@Deprecated
public static final Types.NestedField POSITION_DELETE_FILE_COUNT

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.POSITION_DELETE_FILE_COUNT

EQUALITY_DELETE_RECORD_COUNT

@Deprecated
public static final Types.NestedField EQUALITY_DELETE_RECORD_COUNT

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.EQUALITY_DELETE_RECORD_COUNT

EQUALITY_DELETE_FILE_COUNT

@Deprecated
public static final Types.NestedField EQUALITY_DELETE_FILE_COUNT

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.EQUALITY_DELETE_FILE_COUNT

TOTAL_RECORD_COUNT

@Deprecated
public static final Types.NestedField TOTAL_RECORD_COUNT

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.TOTAL_RECORD_COUNT

LAST_UPDATED_AT

@Deprecated
public static final Types.NestedField LAST_UPDATED_AT

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.LAST_UPDATED_AT

LAST_UPDATED_SNAPSHOT_ID

@Deprecated
public static final Types.NestedField LAST_UPDATED_SNAPSHOT_ID

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.LAST_UPDATED_SNAPSHOT_ID

DV_COUNT

@Deprecated
public static final Types.NestedField DV_COUNT

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.DV_COUNT

Method Details

schema

@Deprecated
public static Schema schema(Types.StructType unifiedPartitionType,
 int formatVersion)

Deprecated.

will be removed in 1.12.0. Use PartitionStatistics.schema(StructType, int) instead.

Generates the partition stats file schema for a given format version based on a combined partition type which considers all specs in a table.

Parameters:

unifiedPartitionType - unified partition schema type. Could be calculated by Partitioning.partitionType(Table).

Returns:

a schema that corresponds to the provided unified partition type.

computeAndWriteStatsFile

public static PartitionStatisticsFile computeAndWriteStatsFile(Table table)
                                                        throws IOException

Computes the stats incrementally after the snapshot that has partition stats file till the current snapshot and writes the combined result into a PartitionStatisticsFile after merging the stats for a given table's current snapshot.

Does a full compute if previous statistics file does not exist.

Parameters:

table - The Table for which the partition statistics is computed.

Returns:

PartitionStatisticsFile for the current snapshot, or null if no statistics are present.

Throws:

IOException

computeAndWriteStatsFile

public static PartitionStatisticsFile computeAndWriteStatsFile(Table table,
 long snapshotId)
                                                        throws IOException

Computes the stats incrementally after the snapshot that has partition stats file till the given snapshot and writes the combined result into a PartitionStatisticsFile after merging the stats for a given snapshot.

Does a full compute if previous statistics file does not exist.

Parameters:

table - The Table for which the partition statistics is computed.

snapshotId - snapshot for which partition statistics are computed.

Returns:

PartitionStatisticsFile for the given snapshot, or null if no statistics are present.

Throws:

IOException

readPartitionStatsFile

@Deprecated
public static CloseableIterable<PartitionStats> readPartitionStatsFile(Schema schema,
 InputFile inputFile)

Deprecated.

will be removed in 1.12.0, use PartitionStatisticsScan instead

Reads partition statistics from the specified InputFile using given schema.

Parameters:

schema - The Schema of the partition statistics file.

inputFile - An InputFile pointing to the partition stats file.