Package org.apache.iceberg.hive

Class HiveTableOperations

java.lang.Object

org.apache.iceberg.BaseMetastoreOperations

org.apache.iceberg.BaseMetastoreTableOperations

org.apache.iceberg.hive.HiveTableOperations

All Implemented Interfaces:

TableOperations

public class HiveTableOperations
extends BaseMetastoreTableOperations

TODO we should be able to extract some more commonalities to BaseMetastoreTableOperations to avoid code duplication between this class and Metacat Tables.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.iceberg.BaseMetastoreOperations

BaseMetastoreOperations.CommitStatus

Field Summary

Fields

Modifier and Type	Field	Description
static final String	HIVE_TABLE_PROPERTY_MAX_SIZE
static final long	HIVE_TABLE_PROPERTY_MAX_SIZE_DEFAULT
static final String	ICEBERG_VIEW_TYPE_VALUE
static final String	NO_LOCK_EXPECTED_KEY
static final String	NO_LOCK_EXPECTED_VALUE

Fields inherited from class org.apache.iceberg.BaseMetastoreTableOperations

ICEBERG_TABLE_TYPE_VALUE, METADATA_HASH_PROP, METADATA_LOCATION_PROP, PREVIOUS_METADATA_LOCATION_PROP, TABLE_TYPE_PROP

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	HiveTableOperations(org.apache.hadoop.conf.Configuration conf, ClientPool<org.apache.hadoop.hive.metastore.IMetaStoreClient,org.apache.thrift.TException> metaClients, FileIO fileIO, KeyManagementClient keyManagementClient, String catalogName, String database, String table)

Method Summary

Modifier and Type	Method	Description
String	database()
protected void	doCommit(TableMetadata base, TableMetadata metadata)
protected void	doRefresh()
EncryptionManager	encryption()	Returns a EncryptionManager to encrypt and decrypt data files.
default Map<String,String>	hmsEnvContext(String metadataLocation)
FileIO	io()	Returns a FileIO to read and write table data and metadata files.
default org.apache.hadoop.hive.metastore.api.Table	loadHmsTable()
long	maxHiveTablePropertySize()
ClientPool<org.apache.hadoop.hive.metastore.IMetaStoreClient,org.apache.thrift.TException>	metaClients()
default org.apache.hadoop.hive.metastore.api.Table	newHmsTable(String hmsTableOwner)
default void	persistTable(org.apache.hadoop.hive.metastore.api.Table hmsTable, boolean updateHiveTable, String metadataLocation)
String	table()
protected String	tableName()	The full name of the table used for logging purposes only.
org.apache.hadoop.hive.metastore.TableType	tableType()
TableOperations	temp(TableMetadata uncommittedMetadata)	Return a temporary TableOperations instance that uses configuration from uncommitted metadata.

Methods inherited from class org.apache.iceberg.BaseMetastoreTableOperations

checkCommitStatus, checkCommitStatusStrict, commit, current, currentMetadataLocation, currentVersion, disableRefresh, locationProvider, metadataFileLocation, metadataFileLocation, refresh, refreshFromMetadataLocation, refreshFromMetadataLocation, refreshFromMetadataLocation, refreshFromMetadataLocation, requestRefresh, writeNewMetadata, writeNewMetadataIfRequired

Methods inherited from class org.apache.iceberg.BaseMetastoreOperations

checkCommitStatus, checkCommitStatusStrict

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.TableOperations

newSnapshotId, requireStrictCleanup

Field Details

HIVE_TABLE_PROPERTY_MAX_SIZE

static final String HIVE_TABLE_PROPERTY_MAX_SIZE

See Also:

• Constant Field Values

HIVE_TABLE_PROPERTY_MAX_SIZE_DEFAULT

static final long HIVE_TABLE_PROPERTY_MAX_SIZE_DEFAULT

See Also:

• Constant Field Values

NO_LOCK_EXPECTED_KEY

static final String NO_LOCK_EXPECTED_KEY

See Also:

• Constant Field Values

NO_LOCK_EXPECTED_VALUE

static final String NO_LOCK_EXPECTED_VALUE

See Also:

• Constant Field Values

ICEBERG_VIEW_TYPE_VALUE

static final String ICEBERG_VIEW_TYPE_VALUE

See Also:

• Constant Field Values

Constructor Details

HiveTableOperations

protected HiveTableOperations(org.apache.hadoop.conf.Configuration conf,
 ClientPool<org.apache.hadoop.hive.metastore.IMetaStoreClient,org.apache.thrift.TException> metaClients,
 FileIO fileIO,
 KeyManagementClient keyManagementClient,
 String catalogName,
 String database,
 String table)

Method Details

tableName

protected String tableName()

Description copied from class: BaseMetastoreTableOperations

The full name of the table used for logging purposes only. For example for HiveTableOperations it is catalogName + "." + database + "." + table.

Specified by:

tableName in class BaseMetastoreTableOperations

Returns:

The full name

io

public FileIO io()

Description copied from interface: TableOperations

Returns a FileIO to read and write table data and metadata files.

Specified by:

io in interface TableOperations

encryption

public EncryptionManager encryption()

Description copied from interface: TableOperations

Returns a EncryptionManager to encrypt and decrypt data files.

Specified by:

encryption in interface TableOperations

doRefresh

protected void doRefresh()

Overrides:

doRefresh in class BaseMetastoreTableOperations

doCommit

protected void doCommit(TableMetadata base,
 TableMetadata metadata)

Overrides:

doCommit in class BaseMetastoreTableOperations

maxHiveTablePropertySize

public long maxHiveTablePropertySize()

database

public String database()

table

public String table()

tableType

public org.apache.hadoop.hive.metastore.TableType tableType()

metaClients

public ClientPool<org.apache.hadoop.hive.metastore.IMetaStoreClient,org.apache.thrift.TException> metaClients()

temp

public TableOperations temp(TableMetadata uncommittedMetadata)

Description copied from interface: TableOperations

Return a temporary TableOperations instance that uses configuration from uncommitted metadata.

This is called by transactions when uncommitted table metadata should be used; for example, to create a metadata file location based on metadata in the transaction that has not been committed.

Transactions will not call TableOperations.refresh() or TableOperations.commit(TableMetadata, TableMetadata).

Specified by:

temp in interface TableOperations

Overrides:

temp in class BaseMetastoreTableOperations

Parameters:

uncommittedMetadata - uncommitted table metadata

Returns:

a temporary table operations that behaves like the uncommitted metadata is current

loadHmsTable

default org.apache.hadoop.hive.metastore.api.Table loadHmsTable()
                                                         throws org.apache.thrift.TException,
InterruptedException

Throws:

org.apache.thrift.TException

InterruptedException

hmsEnvContext

default Map<String,String> hmsEnvContext(String metadataLocation)

persistTable

default void persistTable(org.apache.hadoop.hive.metastore.api.Table hmsTable,
 boolean updateHiveTable,
 String metadataLocation)
                   throws org.apache.thrift.TException,
InterruptedException

Throws:

org.apache.thrift.TException

InterruptedException

newHmsTable

default org.apache.hadoop.hive.metastore.api.Table newHmsTable(String hmsTableOwner)