org.apache.iceberg

Class MetricsUtil

java.lang.Object

org.apache.iceberg.MetricsUtil

public class MetricsUtil
extends java.lang.Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MetricsUtil.ReadableColMetricsStruct A struct of readable metric values for a primitive column
static class	MetricsUtil.ReadableMetricColDefinition Fixed definition of a readable metric column, ie a mapping of a raw metric to a readable metric
static class	MetricsUtil.ReadableMetricsStruct A struct, consisting of all MetricsUtil.ReadableColMetricsStruct for all primitive columns of the table

Field Summary

Fields

Modifier and Type	Field and Description
static java.util.List<MetricsUtil.ReadableMetricColDefinition>	READABLE_METRIC_COLS
static java.lang.String	READABLE_METRICS

Method Summary

Modifier and Type	Method and Description
static Metrics	copyWithoutFieldCounts(Metrics metrics, java.util.Set<java.lang.Integer> excludedFieldIds) Copies a metrics object without value, NULL and NaN counts for given fields.
static Metrics	copyWithoutFieldCountsAndBounds(Metrics metrics, java.util.Set<java.lang.Integer> excludedFieldIds) Copies a metrics object without counts and bounds for given fields.
static java.util.Map<java.lang.Integer,java.lang.Long>	createNanValueCounts(java.util.stream.Stream<FieldMetrics<?>> fieldMetrics, MetricsConfig metricsConfig, Schema inputSchema) Construct mapping relationship between column id to NaN value counts from input metrics and metrics config.
static MetricsModes.MetricsMode	metricsMode(Schema inputSchema, MetricsConfig metricsConfig, int fieldId) Extract MetricsMode for the given field id from metrics config.
static Schema	readableMetricsSchema(Schema dataTableSchema, Schema metadataTableSchema) Calculates a dynamic schema for readable_metrics to add to metadata tables.
static MetricsUtil.ReadableMetricsStruct	readableMetricsStruct(Schema schema, ContentFile<?> file, Types.StructType projectedSchema) Return a readable metrics struct row from file metadata

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

READABLE_METRIC_COLS

public static final java.util.List<MetricsUtil.ReadableMetricColDefinition> READABLE_METRIC_COLS

READABLE_METRICS

public static final java.lang.String READABLE_METRICS

See Also:

Constant Field Values

Method Detail

copyWithoutFieldCounts

public static Metrics copyWithoutFieldCounts(Metrics metrics,
                                             java.util.Set<java.lang.Integer> excludedFieldIds)

Copies a metrics object without value, NULL and NaN counts for given fields.

Parameters:

excludedFieldIds - field IDs for which the counts must be dropped

Returns:

a new metrics object without counts for given fields

copyWithoutFieldCountsAndBounds

public static Metrics copyWithoutFieldCountsAndBounds(Metrics metrics,
                                                      java.util.Set<java.lang.Integer> excludedFieldIds)

Copies a metrics object without counts and bounds for given fields.

Parameters:

excludedFieldIds - field IDs for which the counts and bounds must be dropped

Returns:

a new metrics object without lower and upper bounds for given fields

createNanValueCounts

public static java.util.Map<java.lang.Integer,java.lang.Long> createNanValueCounts(java.util.stream.Stream<FieldMetrics<?>> fieldMetrics,
                                                                                   MetricsConfig metricsConfig,
                                                                                   Schema inputSchema)

Construct mapping relationship between column id to NaN value counts from input metrics and metrics config.

metricsMode

public static MetricsModes.MetricsMode metricsMode(Schema inputSchema,
                                                   MetricsConfig metricsConfig,
                                                   int fieldId)

Extract MetricsMode for the given field id from metrics config.

readableMetricsSchema

public static Schema readableMetricsSchema(Schema dataTableSchema,
                                           Schema metadataTableSchema)

Calculates a dynamic schema for readable_metrics to add to metadata tables. The type will be the struct MetricsUtil.ReadableColMetricsStruct, composed of MetricsUtil.ReadableMetricsStruct for all primitive columns in the data table

Parameters:

dataTableSchema - schema of data table

metadataTableSchema - schema of existing metadata table (to ensure id uniqueness)

Returns:

schema of readable_metrics struct

readableMetricsStruct

public static MetricsUtil.ReadableMetricsStruct readableMetricsStruct(Schema schema,
                                                                      ContentFile<?> file,
                                                                      Types.StructType projectedSchema)

Return a readable metrics struct row from file metadata

Parameters:

schema - schema of original data table

file - content file with metrics

projectedSchema - user requested projection

Returns:

MetricsUtil.ReadableMetricsStruct