Package org.apache.iceberg.util

Class ThreadPools

  • public class ThreadPools
extends java.lang.Object

    • Method Summary

      All Methods Static Methods Concrete Methods Deprecated Methods 
      Modifier and Type Method Description
      static java.util.concurrent.ExecutorService getDeleteWorkerPool()
      Return an ExecutorService that uses the "delete worker" thread-pool.
      static java.util.concurrent.ExecutorService getWorkerPool()
      Return an ExecutorService that uses the "worker" thread-pool.
      static java.util.concurrent.ExecutorService newExitingWorkerPool​(java.lang.String namePrefix, int poolSize)
      Creates a fixed-size thread pool that uses daemon threads and registers a shutdown hook to ensure the pool terminates when the JVM exits.
      static java.util.concurrent.ExecutorService newFixedThreadPool​(java.lang.String namePrefix, int poolSize)
      Creates a fixed-size thread pool that uses daemon threads.
      static java.util.concurrent.ScheduledExecutorService newScheduledPool​(java.lang.String namePrefix, int poolSize)
      Create a new ScheduledExecutorService with the given name and pool size.
      static java.util.concurrent.ExecutorService newWorkerPool​(java.lang.String namePrefix)
      Deprecated.
      will be removed in 2.0.0.
      static java.util.concurrent.ExecutorService newWorkerPool​(java.lang.String namePrefix, int poolSize)
      Deprecated.
      will be removed in 2.0.0.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • WORKER_THREAD_POOL_SIZE_PROP

        @Deprecated
public static final java.lang.String WORKER_THREAD_POOL_SIZE_PROP
        Deprecated.
        Use WORKER_THREAD_POOL_SIZE instead; will be removed in 2.0.0

      • WORKER_THREAD_POOL_SIZE

        public static final int WORKER_THREAD_POOL_SIZE

      • DELETE_WORKER_THREAD_POOL_SIZE

        public static final int DELETE_WORKER_THREAD_POOL_SIZE

    • Method Detail

      • getWorkerPool

        public static java.util.concurrent.ExecutorService getWorkerPool()
        Return an ExecutorService that uses the "worker" thread-pool.

        The size of the worker pool limits the number of tasks concurrently reading manifests in the base table implementation across all concurrent planning operations.

        The size of this thread-pool is controlled by the Java system property iceberg.worker.num-threads.

        Returns:
        an ExecutorService that uses the worker pool

      • getDeleteWorkerPool

        public static java.util.concurrent.ExecutorService getDeleteWorkerPool()
        Return an ExecutorService that uses the "delete worker" thread-pool.

        The size of this worker pool limits the number of tasks concurrently reading delete files within a single JVM. If there are multiple threads loading deletes, all of them will share this worker pool by default.

        The size of this thread-pool is controlled by the Java system property iceberg.worker.delete-num-threads.

        Returns:
        an ExecutorService that uses the delete worker pool

      • newWorkerPool

        @Deprecated
public static java.util.concurrent.ExecutorService newWorkerPool​(java.lang.String namePrefix)
        Deprecated.
        will be removed in 2.0.0. Use newExitingWorkerPool(String, int) for long-lived thread pools that require a shutdown hook, or newFixedThreadPool(String, int) for short-lived thread pools where you manage the lifecycle.
        Creates a fixed-size thread pool that uses daemon threads. The pool is wrapped with MoreExecutors.getExitingExecutorService(ThreadPoolExecutor), which registers a shutdown hook to ensure the pool terminates when the JVM exits. Important: Even if the pool is explicitly shut down using ExecutorService.shutdown(), the shutdown hook is not removed. This can lead to accumulation of shutdown hooks if this method is used repeatedly for short-lived thread pools.

        For clarity and to avoid potential issues with shutdown hook accumulation, prefer using either newExitingWorkerPool(String, int) or newFixedThreadPool(String, int), depending on the intended lifecycle of the thread pool.

      • newWorkerPool

        @Deprecated
public static java.util.concurrent.ExecutorService newWorkerPool​(java.lang.String namePrefix,
                                                                 int poolSize)
        Deprecated.
        will be removed in 2.0.0. Use newExitingWorkerPool(String, int) for long-lived thread pools that require a shutdown hook, or newFixedThreadPool(String, int) for short-lived thread pools where you manage the lifecycle.
        Creates a fixed-size thread pool that uses daemon threads. The pool is wrapped with MoreExecutors.getExitingExecutorService(ThreadPoolExecutor), which registers a shutdown hook to ensure the pool terminates when the JVM exits. Important: Even if the pool is explicitly shut down using ExecutorService.shutdown(), the shutdown hook is not removed. This can lead to accumulation of shutdown hooks if this method is used repeatedly for short-lived thread pools.

        For clarity and to avoid potential issues with shutdown hook accumulation, prefer using either newExitingWorkerPool(String, int) or newFixedThreadPool(String, int), depending on the intended lifecycle of the thread pool.

      • newExitingWorkerPool

        public static java.util.concurrent.ExecutorService newExitingWorkerPool​(java.lang.String namePrefix,
                                                                        int poolSize)
        Creates a fixed-size thread pool that uses daemon threads and registers a shutdown hook to ensure the pool terminates when the JVM exits. This is suitable for long-lived thread pools that should be automatically cleaned up on JVM shutdown.

      • newFixedThreadPool

        public static java.util.concurrent.ExecutorService newFixedThreadPool​(java.lang.String namePrefix,
                                                                      int poolSize)
        Creates a fixed-size thread pool that uses daemon threads.

      • newScheduledPool

        public static java.util.concurrent.ScheduledExecutorService newScheduledPool​(java.lang.String namePrefix,
                                                                             int poolSize)
        Create a new ScheduledExecutorService with the given name and pool size.

        Threads used by this service will be daemon threads.

        Parameters:
        namePrefix - a base name for threads in the executor service's thread pool
        poolSize - max number of threads to use
        Returns:
        an executor service