Package org.apache.iceberg

Class RewriteTablePathUtil

java.lang.Object

org.apache.iceberg.RewriteTablePathUtil

public class RewriteTablePathUtil
extends Object

Utilities for Rewrite table path action.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	RewriteTablePathUtil.PositionDeleteReaderWriter	Class providing engine-specific methods to read and write position delete files.
static class	RewriteTablePathUtil.RewriteResult<T>	Rewrite result.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	FILE_SEPARATOR

Method Summary

Modifier and Type	Method	Description
static String	combinePaths(String absolutePath, String relativePath)	Combine a base and relative path.
static String	fileName(String path)	Returns the file name of a path.
static String	maybeAppendFileSeparator(String path)
static String	newPath(String path, String sourcePrefix, String targetPrefix)	Replace path reference
static String	relativize(String path, String prefix)	Relativize a path.
static TableMetadata	replacePaths(TableMetadata metadata, String sourcePrefix, String targetPrefix)	Create a new table metadata object, replacing path references
static RewriteTablePathUtil.RewriteResult<DataFile>	rewriteDataManifest(ManifestFile manifestFile, Set<Long> snapshotIds, OutputFile outputFile, FileIO io, int format, Map<Integer,PartitionSpec> specsById, String sourcePrefix, String targetPrefix)	Rewrite a data manifest, replacing path references.
static RewriteTablePathUtil.RewriteResult<DataFile>	rewriteDataManifest(ManifestFile manifestFile, OutputFile outputFile, FileIO io, int format, Map<Integer,PartitionSpec> specsById, String sourcePrefix, String targetPrefix)	Deprecated. since 1.10.0, will be removed in 1.11.0
static RewriteTablePathUtil.RewriteResult<DeleteFile>	rewriteDeleteManifest(ManifestFile manifestFile, Set<Long> snapshotIds, OutputFile outputFile, FileIO io, int format, Map<Integer,PartitionSpec> specsById, String sourcePrefix, String targetPrefix, String stagingLocation)	Rewrite a delete manifest, replacing path references.
static RewriteTablePathUtil.RewriteResult<DeleteFile>	rewriteDeleteManifest(ManifestFile manifestFile, OutputFile outputFile, FileIO io, int format, Map<Integer,PartitionSpec> specsById, String sourcePrefix, String targetPrefix, String stagingLocation)	Deprecated. since 1.10.0, will be removed in 1.11.0
static RewriteTablePathUtil.RewriteResult<ManifestFile>	rewriteManifestList(Snapshot snapshot, FileIO io, TableMetadata tableMetadata, Set<String> manifestsToRewrite, String sourcePrefix, String targetPrefix, String stagingDir, String outputPath)	Rewrite a manifest list representing a snapshot, replacing path references.
static void	rewritePositionDeleteFile(DeleteFile deleteFile, OutputFile outputFile, FileIO io, PartitionSpec spec, String sourcePrefix, String targetPrefix, RewriteTablePathUtil.PositionDeleteReaderWriter posDeleteReaderWriter)	Rewrite a position delete file, replacing path references.
static String	stagingPath(String originalPath, String stagingDir)	Deprecated. since 1.10.0, will be removed in 1.11.0.
static String	stagingPath(String originalPath, String sourcePrefix, String stagingDir)	Construct a staging path under a given staging directory, preserving relative directory structure to avoid conflicts when multiple files have the same name but different paths.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

FILE_SEPARATOR

public static final String FILE_SEPARATOR

See Also:

• Constant Field Values

Method Details

replacePaths

public static TableMetadata replacePaths(TableMetadata metadata,
 String sourcePrefix,
 String targetPrefix)

Create a new table metadata object, replacing path references

Parameters:

metadata - source table metadata

sourcePrefix - source prefix that will be replaced

targetPrefix - target prefix that will replace it

Returns:

copy of table metadata with paths replaced

rewriteManifestList

public static RewriteTablePathUtil.RewriteResult<ManifestFile> rewriteManifestList(Snapshot snapshot,
 FileIO io,
 TableMetadata tableMetadata,
 Set<String> manifestsToRewrite,
 String sourcePrefix,
 String targetPrefix,
 String stagingDir,
 String outputPath)

Rewrite a manifest list representing a snapshot, replacing path references.

Parameters:

snapshot - snapshot represented by the manifest list

io - file io

tableMetadata - metadata of table

manifestsToRewrite - a list of manifest files to filter for rewrite

sourcePrefix - source prefix that will be replaced

targetPrefix - target prefix that will replace it

stagingDir - staging directory

outputPath - location to write the manifest list

Returns:

a copy plan for manifest files whose metadata were contained in the rewritten manifest list

rewriteDataManifest

@Deprecated
public static RewriteTablePathUtil.RewriteResult<DataFile> rewriteDataManifest(ManifestFile manifestFile,
 OutputFile outputFile,
 FileIO io,
 int format,
 Map<Integer,PartitionSpec> specsById,
 String sourcePrefix,
 String targetPrefix)
                                                                        throws IOException

Deprecated.

since 1.10.0, will be removed in 1.11.0

Rewrite a data manifest, replacing path references.

Parameters:

manifestFile - source manifest file to rewrite

outputFile - output file to rewrite manifest file to

io - file io

format - format of the manifest file

specsById - map of partition specs by id

sourcePrefix - source prefix that will be replaced

targetPrefix - target prefix that will replace it

Returns:

a copy plan of content files in the manifest that was rewritten

Throws:

IOException

rewriteDataManifest

public static RewriteTablePathUtil.RewriteResult<DataFile> rewriteDataManifest(ManifestFile manifestFile,
 Set<Long> snapshotIds,
 OutputFile outputFile,
 FileIO io,
 int format,
 Map<Integer,PartitionSpec> specsById,
 String sourcePrefix,
 String targetPrefix)
                                                                        throws IOException

Rewrite a data manifest, replacing path references.

Parameters:

manifestFile - source manifest file to rewrite

snapshotIds - snapshot ids for filtering returned data manifest entries

outputFile - output file to rewrite manifest file to

io - file io

format - format of the manifest file

specsById - map of partition specs by id

sourcePrefix - source prefix that will be replaced

targetPrefix - target prefix that will replace it

Returns:

a copy plan of content files in the manifest that was rewritten

Throws:

IOException

rewriteDeleteManifest

@Deprecated
public static RewriteTablePathUtil.RewriteResult<DeleteFile> rewriteDeleteManifest(ManifestFile manifestFile,
 OutputFile outputFile,
 FileIO io,
 int format,
 Map<Integer,PartitionSpec> specsById,
 String sourcePrefix,
 String targetPrefix,
 String stagingLocation)
                                                                            throws IOException

Deprecated.

since 1.10.0, will be removed in 1.11.0

Rewrite a delete manifest, replacing path references.

Parameters:

manifestFile - source delete manifest to rewrite

outputFile - output file to rewrite manifest file to

io - file io

format - format of the manifest file

specsById - map of partition specs by id

sourcePrefix - source prefix that will be replaced

targetPrefix - target prefix that will replace it

stagingLocation - staging location for rewritten files (referred delete file will be rewritten here)

Returns:

a copy plan of content files in the manifest that was rewritten

Throws:

IOException

rewriteDeleteManifest

public static RewriteTablePathUtil.RewriteResult<DeleteFile> rewriteDeleteManifest(ManifestFile manifestFile,
 Set<Long> snapshotIds,
 OutputFile outputFile,
 FileIO io,
 int format,
 Map<Integer,PartitionSpec> specsById,
 String sourcePrefix,
 String targetPrefix,
 String stagingLocation)
                                                                            throws IOException

Rewrite a delete manifest, replacing path references.

Parameters:

manifestFile - source delete manifest to rewrite

snapshotIds - snapshot ids for filtering returned delete manifest entries

outputFile - output file to rewrite manifest file to

io - file io

format - format of the manifest file

specsById - map of partition specs by id

sourcePrefix - source prefix that will be replaced

targetPrefix - target prefix that will replace it

stagingLocation - staging location for rewritten files (referred delete file will be rewritten here)

Returns:

a copy plan of content files in the manifest that was rewritten

Throws:

IOException

rewritePositionDeleteFile

public static void rewritePositionDeleteFile(DeleteFile deleteFile,
 OutputFile outputFile,
 FileIO io,
 PartitionSpec spec,
 String sourcePrefix,
 String targetPrefix,
 RewriteTablePathUtil.PositionDeleteReaderWriter posDeleteReaderWriter)
                                      throws IOException

Rewrite a position delete file, replacing path references.

Parameters:

deleteFile - source delete file to be rewritten

outputFile - output file to rewrite delete file to

io - file io

spec - spec of delete file

sourcePrefix - source prefix that will be replaced

targetPrefix - target prefix to replace it

posDeleteReaderWriter - class to read and write position delete files

Throws:

IOException

newPath

public static String newPath(String path,
 String sourcePrefix,
 String targetPrefix)

Replace path reference

Parameters:

path - path reference

sourcePrefix - source prefix that will be replaced

targetPrefix - target prefix that will replace it

Returns:

new path reference

combinePaths

public static String combinePaths(String absolutePath,
 String relativePath)

Combine a base and relative path.

fileName

public static String fileName(String path)

Returns the file name of a path.

relativize

public static String relativize(String path,
 String prefix)

Relativize a path.

maybeAppendFileSeparator

public static String maybeAppendFileSeparator(String path)

stagingPath

@Deprecated
public static String stagingPath(String originalPath,
 String stagingDir)

Deprecated.

since 1.10.0, will be removed in 1.11.0. Use stagingPath(String, String, String) instead to avoid filename conflicts

Construct a staging path under a given staging directory

Parameters:

originalPath - source path

stagingDir - staging directory

Returns:

a staging path under the staging directory, based on the original path

stagingPath

public static String stagingPath(String originalPath,
 String sourcePrefix,
 String stagingDir)

Construct a staging path under a given staging directory, preserving relative directory structure to avoid conflicts when multiple files have the same name but different paths.

Parameters:

originalPath - source path

sourcePrefix - source prefix to be replaced

stagingDir - staging directory

Returns:

a staging path under the staging directory that preserves the relative path structure