org.apache.iceberg.spark

Class SparkUtil

java.lang.Object

org.apache.iceberg.spark.SparkUtil

public class SparkUtil
extends java.lang.Object

Method Summary

Modifier and Type	Method and Description
static <C,T> Pair<C,T>	catalogAndIdentifier(java.util.List<java.lang.String> nameParts, java.util.function.Function<java.lang.String,C> catalogProvider, java.util.function.BiFunction<java.lang.String[],java.lang.String,T> identiferProvider, C currentCatalog, java.lang.String[] currentNamespace) A modified version of Spark's LookupCatalog.CatalogAndIdentifier.unapply Attempts to find the catalog and identifier a multipart identifier represents
static FileIO	serializableFileIO(Table table)
static void	validatePartitionTransforms(PartitionSpec spec) Check whether the partition transforms in a spec can be used to write data.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

serializableFileIO

public static FileIO serializableFileIO(Table table)

validatePartitionTransforms

public static void validatePartitionTransforms(PartitionSpec spec)

Check whether the partition transforms in a spec can be used to write data.

Parameters:

spec - a PartitionSpec

Throws:

java.lang.UnsupportedOperationException - if the spec contains unknown partition transforms

catalogAndIdentifier

public static <C,T> Pair<C,T> catalogAndIdentifier(java.util.List<java.lang.String> nameParts,
                                                   java.util.function.Function<java.lang.String,C> catalogProvider,
                                                   java.util.function.BiFunction<java.lang.String[],java.lang.String,T> identiferProvider,
                                                   C currentCatalog,
                                                   java.lang.String[] currentNamespace)

A modified version of Spark's LookupCatalog.CatalogAndIdentifier.unapply Attempts to find the catalog and identifier a multipart identifier represents

Parameters:

nameParts - Multipart identifier representing a table

Returns:

The CatalogPlugin and Identifier for the table