Package org.apache.iceberg.mr
Class InputFormatConfig
- java.lang.Object
-
- org.apache.iceberg.mr.InputFormatConfig
-
public class InputFormatConfig extends java.lang.Object
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
InputFormatConfig.ConfigBuilder
static class
InputFormatConfig.InMemoryDataModel
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
AS_OF_TIMESTAMP
static java.lang.String
CASE_SENSITIVE
static boolean
CASE_SENSITIVE_DEFAULT
static java.lang.String
CATALOG
Deprecated.please usecatalogPropertyConfigKey(String, String)
with config keyCatalogUtil.ICEBERG_CATALOG_TYPE
to specify the type of a catalog.static java.lang.String
CATALOG_CONFIG_PREFIX
static java.lang.String
CATALOG_LOADER_CLASS
Deprecated.please usecatalogPropertyConfigKey(String, String)
with config keyCatalogProperties.CATALOG_IMPL
to specify the implementation of a catalog.static java.lang.String
CATALOG_NAME
static java.lang.String
COMMIT_FILE_THREAD_POOL_SIZE
static int
COMMIT_FILE_THREAD_POOL_SIZE_DEFAULT
static java.lang.String
COMMIT_TABLE_THREAD_POOL_SIZE
static int
COMMIT_TABLE_THREAD_POOL_SIZE_DEFAULT
static java.lang.String
CONFIG_SERIALIZATION_DISABLED
static boolean
CONFIG_SERIALIZATION_DISABLED_DEFAULT
static java.lang.String
EXTERNAL_TABLE_PURGE
static java.lang.String
FILTER_EXPRESSION
static java.lang.String
HADOOP_CATALOG
static java.lang.String
HADOOP_CATALOG_WAREHOUSE_LOCATION
Deprecated.please usecatalogPropertyConfigKey(String, String)
with config keyCatalogProperties.WAREHOUSE_LOCATION
to specify the warehouse location of a catalog.static java.lang.String
HADOOP_TABLES
static java.lang.String
HIVE_CATALOG
static java.lang.String
ICEBERG_SNAPSHOTS_TABLE_SUFFIX
static java.lang.String
IN_MEMORY_DATA_MODEL
static java.lang.String
LOCALITY
static java.lang.String
OUTPUT_TABLES
static java.lang.String
PARTITION_SPEC
static java.lang.String
READ_SCHEMA
static java.lang.String
REUSE_CONTAINERS
static java.lang.String
SCHEMA_AUTO_CONVERSION
static java.lang.String
SELECTED_COLUMNS
static java.lang.String
SERIALIZED_TABLE_PREFIX
static java.lang.String
SKIP_RESIDUAL_FILTERING
static java.lang.String
SNAPSHOT_ID
static java.lang.String
SNAPSHOT_TABLE
static java.lang.String
SNAPSHOT_TABLE_SUFFIX
static java.lang.String
SPLIT_SIZE
static java.lang.String
TABLE_CATALOG_PREFIX
static java.lang.String
TABLE_IDENTIFIER
static java.lang.String
TABLE_LOCATION
static java.lang.String
TABLE_SCHEMA
static java.lang.String
WRITE_TARGET_FILE_SIZE
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static java.lang.String
catalogPropertyConfigKey(java.lang.String catalogName, java.lang.String catalogProperty)
Get Hadoop config key of a catalog property based on catalog namestatic Schema
readSchema(org.apache.hadoop.conf.Configuration conf)
static java.lang.String[]
selectedColumns(org.apache.hadoop.conf.Configuration conf)
static Schema
tableSchema(org.apache.hadoop.conf.Configuration conf)
-
-
-
Field Detail
-
REUSE_CONTAINERS
public static final java.lang.String REUSE_CONTAINERS
- See Also:
- Constant Field Values
-
SKIP_RESIDUAL_FILTERING
public static final java.lang.String SKIP_RESIDUAL_FILTERING
- See Also:
- Constant Field Values
-
AS_OF_TIMESTAMP
public static final java.lang.String AS_OF_TIMESTAMP
- See Also:
- Constant Field Values
-
FILTER_EXPRESSION
public static final java.lang.String FILTER_EXPRESSION
- See Also:
- Constant Field Values
-
IN_MEMORY_DATA_MODEL
public static final java.lang.String IN_MEMORY_DATA_MODEL
- See Also:
- Constant Field Values
-
READ_SCHEMA
public static final java.lang.String READ_SCHEMA
- See Also:
- Constant Field Values
-
SNAPSHOT_ID
public static final java.lang.String SNAPSHOT_ID
- See Also:
- Constant Field Values
-
SPLIT_SIZE
public static final java.lang.String SPLIT_SIZE
- See Also:
- Constant Field Values
-
SCHEMA_AUTO_CONVERSION
public static final java.lang.String SCHEMA_AUTO_CONVERSION
- See Also:
- Constant Field Values
-
TABLE_IDENTIFIER
public static final java.lang.String TABLE_IDENTIFIER
- See Also:
- Constant Field Values
-
TABLE_LOCATION
public static final java.lang.String TABLE_LOCATION
- See Also:
- Constant Field Values
-
TABLE_SCHEMA
public static final java.lang.String TABLE_SCHEMA
- See Also:
- Constant Field Values
-
PARTITION_SPEC
public static final java.lang.String PARTITION_SPEC
- See Also:
- Constant Field Values
-
SERIALIZED_TABLE_PREFIX
public static final java.lang.String SERIALIZED_TABLE_PREFIX
- See Also:
- Constant Field Values
-
TABLE_CATALOG_PREFIX
public static final java.lang.String TABLE_CATALOG_PREFIX
- See Also:
- Constant Field Values
-
LOCALITY
public static final java.lang.String LOCALITY
- See Also:
- Constant Field Values
-
CATALOG
@Deprecated public static final java.lang.String CATALOG
Deprecated.please usecatalogPropertyConfigKey(String, String)
with config keyCatalogUtil.ICEBERG_CATALOG_TYPE
to specify the type of a catalog.- See Also:
- Constant Field Values
-
HADOOP_CATALOG_WAREHOUSE_LOCATION
@Deprecated public static final java.lang.String HADOOP_CATALOG_WAREHOUSE_LOCATION
Deprecated.please usecatalogPropertyConfigKey(String, String)
with config keyCatalogProperties.WAREHOUSE_LOCATION
to specify the warehouse location of a catalog.- See Also:
- Constant Field Values
-
CATALOG_LOADER_CLASS
@Deprecated public static final java.lang.String CATALOG_LOADER_CLASS
Deprecated.please usecatalogPropertyConfigKey(String, String)
with config keyCatalogProperties.CATALOG_IMPL
to specify the implementation of a catalog.- See Also:
- Constant Field Values
-
SELECTED_COLUMNS
public static final java.lang.String SELECTED_COLUMNS
- See Also:
- Constant Field Values
-
EXTERNAL_TABLE_PURGE
public static final java.lang.String EXTERNAL_TABLE_PURGE
- See Also:
- Constant Field Values
-
CONFIG_SERIALIZATION_DISABLED
public static final java.lang.String CONFIG_SERIALIZATION_DISABLED
- See Also:
- Constant Field Values
-
CONFIG_SERIALIZATION_DISABLED_DEFAULT
public static final boolean CONFIG_SERIALIZATION_DISABLED_DEFAULT
- See Also:
- Constant Field Values
-
OUTPUT_TABLES
public static final java.lang.String OUTPUT_TABLES
- See Also:
- Constant Field Values
-
COMMIT_TABLE_THREAD_POOL_SIZE
public static final java.lang.String COMMIT_TABLE_THREAD_POOL_SIZE
- See Also:
- Constant Field Values
-
COMMIT_TABLE_THREAD_POOL_SIZE_DEFAULT
public static final int COMMIT_TABLE_THREAD_POOL_SIZE_DEFAULT
- See Also:
- Constant Field Values
-
COMMIT_FILE_THREAD_POOL_SIZE
public static final java.lang.String COMMIT_FILE_THREAD_POOL_SIZE
- See Also:
- Constant Field Values
-
COMMIT_FILE_THREAD_POOL_SIZE_DEFAULT
public static final int COMMIT_FILE_THREAD_POOL_SIZE_DEFAULT
- See Also:
- Constant Field Values
-
WRITE_TARGET_FILE_SIZE
public static final java.lang.String WRITE_TARGET_FILE_SIZE
- See Also:
- Constant Field Values
-
CASE_SENSITIVE
public static final java.lang.String CASE_SENSITIVE
- See Also:
- Constant Field Values
-
CASE_SENSITIVE_DEFAULT
public static final boolean CASE_SENSITIVE_DEFAULT
- See Also:
- Constant Field Values
-
CATALOG_NAME
public static final java.lang.String CATALOG_NAME
- See Also:
- Constant Field Values
-
HADOOP_CATALOG
public static final java.lang.String HADOOP_CATALOG
- See Also:
- Constant Field Values
-
HADOOP_TABLES
public static final java.lang.String HADOOP_TABLES
- See Also:
- Constant Field Values
-
HIVE_CATALOG
public static final java.lang.String HIVE_CATALOG
- See Also:
- Constant Field Values
-
ICEBERG_SNAPSHOTS_TABLE_SUFFIX
public static final java.lang.String ICEBERG_SNAPSHOTS_TABLE_SUFFIX
- See Also:
- Constant Field Values
-
SNAPSHOT_TABLE
public static final java.lang.String SNAPSHOT_TABLE
- See Also:
- Constant Field Values
-
SNAPSHOT_TABLE_SUFFIX
public static final java.lang.String SNAPSHOT_TABLE_SUFFIX
- See Also:
- Constant Field Values
-
CATALOG_CONFIG_PREFIX
public static final java.lang.String CATALOG_CONFIG_PREFIX
- See Also:
- Constant Field Values
-
-
Method Detail
-
tableSchema
public static Schema tableSchema(org.apache.hadoop.conf.Configuration conf)
-
readSchema
public static Schema readSchema(org.apache.hadoop.conf.Configuration conf)
-
selectedColumns
public static java.lang.String[] selectedColumns(org.apache.hadoop.conf.Configuration conf)
-
catalogPropertyConfigKey
public static java.lang.String catalogPropertyConfigKey(java.lang.String catalogName, java.lang.String catalogProperty)
Get Hadoop config key of a catalog property based on catalog name- Parameters:
catalogName
- catalog namecatalogProperty
- catalog property, can be any custom property, a commonly used list of properties can be found atCatalogProperties
- Returns:
- Hadoop config key of a catalog property for the catalog name
-
-