Package org.apache.iceberg.util

Class ThreadPools

java.lang.Object
org.apache.iceberg.util.ThreadPools
public class ThreadPools extends Object

  • Field Details

    • WORKER_THREAD_POOL_SIZE_PROP

      @Deprecated public static final String WORKER_THREAD_POOL_SIZE_PROP
      Deprecated.
      Use WORKER_THREAD_POOL_SIZE instead; will be removed in 2.0.0

    • WORKER_THREAD_POOL_SIZE

      public static final int WORKER_THREAD_POOL_SIZE

    • DELETE_WORKER_THREAD_POOL_SIZE

      public static final int DELETE_WORKER_THREAD_POOL_SIZE

    • AUTH_REFRESH_THREAD_POOL_SIZE

      public static final int AUTH_REFRESH_THREAD_POOL_SIZE

  • Method Details

    • getWorkerPool

      public static ExecutorService getWorkerPool()
      Return an ExecutorService that uses the "worker" thread-pool.

      The size of the worker pool limits the number of tasks concurrently reading manifests in the base table implementation across all concurrent planning operations.

      The size of this thread-pool is controlled by the Java system property iceberg.worker.num-threads.

      Returns:
      an ExecutorService that uses the worker pool

    • getDeleteWorkerPool

      public static ExecutorService getDeleteWorkerPool()
      Return an ExecutorService that uses the "delete worker" thread-pool.

      The size of this worker pool limits the number of tasks concurrently reading delete files within a single JVM. If there are multiple threads loading deletes, all of them will share this worker pool by default.

      The size of this thread-pool is controlled by the Java system property iceberg.worker.delete-num-threads.

      Returns:
      an ExecutorService that uses the delete worker pool

    • authRefreshPool

      public static ScheduledExecutorService authRefreshPool()
      A shared ScheduledExecutorService that REST catalogs can use for refreshing their authentication data.

    • newWorkerPool

      @Deprecated public static ExecutorService newWorkerPool(String namePrefix)
      Deprecated.
      will be removed in 2.0.0. Use newExitingWorkerPool(String, int) for long-lived thread pools that require a shutdown hook, or newFixedThreadPool(String, int) for short-lived thread pools where you manage the lifecycle.
      Creates a fixed-size thread pool that uses daemon threads. The pool is wrapped with MoreExecutors.getExitingExecutorService(ThreadPoolExecutor), which registers a shutdown hook to ensure the pool terminates when the JVM exits. Important: Even if the pool is explicitly shut down using ExecutorService.shutdown(), the shutdown hook is not removed. This can lead to accumulation of shutdown hooks if this method is used repeatedly for short-lived thread pools.

      For clarity and to avoid potential issues with shutdown hook accumulation, prefer using either newExitingWorkerPool(String, int) or newFixedThreadPool(String, int), depending on the intended lifecycle of the thread pool.

    • newWorkerPool

      @Deprecated public static ExecutorService newWorkerPool(String namePrefix, int poolSize)
      Deprecated.
      will be removed in 2.0.0. Use newExitingWorkerPool(String, int) for long-lived thread pools that require a shutdown hook, or newFixedThreadPool(String, int) for short-lived thread pools where you manage the lifecycle.
      Creates a fixed-size thread pool that uses daemon threads. The pool is wrapped with MoreExecutors.getExitingExecutorService(ThreadPoolExecutor), which registers a shutdown hook to ensure the pool terminates when the JVM exits. Important: Even if the pool is explicitly shut down using ExecutorService.shutdown(), the shutdown hook is not removed. This can lead to accumulation of shutdown hooks if this method is used repeatedly for short-lived thread pools.

      For clarity and to avoid potential issues with shutdown hook accumulation, prefer using either newExitingWorkerPool(String, int) or newFixedThreadPool(String, int), depending on the intended lifecycle of the thread pool.

    • newExitingWorkerPool

      public static ExecutorService newExitingWorkerPool(String namePrefix, int poolSize)
      Creates a fixed-size thread pool that uses daemon threads and registers a shutdown hook to ensure the pool terminates when the JVM exits. This is suitable for long-lived thread pools that should be automatically cleaned up on JVM shutdown.

    • newFixedThreadPool

      public static ExecutorService newFixedThreadPool(String namePrefix, int poolSize)
      Creates a fixed-size thread pool that uses daemon threads.

    • newScheduledPool

      public static ScheduledExecutorService newScheduledPool(String namePrefix, int poolSize)
      Create a new ScheduledExecutorService with the given name and pool size.

      Threads used by this service will be daemon threads.

      Parameters:
      namePrefix - a base name for threads in the executor service's thread pool
      poolSize - max number of threads to use
      Returns:
      an executor service

    • newExitingScheduledPool

      public static ScheduledExecutorService newExitingScheduledPool(String namePrefix, int poolSize, Duration terminationTimeout)
      Create a new ScheduledExecutorService with the given name and pool size.

      Threads used by this service will be daemon threads.

      The service registers a shutdown hook to ensure that it terminates when the JVM exits. This is suitable for long-lived thread pools that should be automatically cleaned up on JVM shutdown.