Thread Pools SE 441 Prof. Bullinger
Thread Pools Thread Pool Limitations Sizing Thread Pools ThreadPoolExecutor Queuing Tasks Parallelizing Loops
Thread Pool Limitations Task Dependencies Different types of tasks require different execution policies Independent tasks are ideal Dependent tasks induce coupling Timing Results Side-effects Other tasks
Thread Confinement Thread confinement couples a task to its execution policy A single-threaded Executor does not require a thread-safe task Changing to a multi-threaded Executor introduces potential thread-safety issues Time-sensitive tasks may be impeded by an execution policy A long task in a single-threaded Executor may affect response time of other tasks
ThreadLocal ThreadLocal provides thread-specific state Executors reuse threads, rendering thread-specific state unusable Threads may also come and go, depending on the execution policy of the Executor Limit ThreadLocal to the lifecycle of a Task
Thread Pool Heuristics Thread Pools work best when tasks are homogenous and independent Don t mix tasks that depend on each other in the same Executor One task blocks, waiting for the results of the other The other cannot run due to limited resources A single-threaded thread pool A thread-pool with no available threads
Thread Pool Heuristics Thread Pools work best when tasks are homogenous and independent Don t mix long-running and short-running tasks Eventually, long-running tasks dominate the thread pool Responsiveness may suffer Suggest adding timeouts to blocking calls to ensure liveness Large or unbounded thread pools minimize risk Document the execution policy requirement for a task!
Thread Pools Thread Pool Limitations Sizing Thread Pools ThreadPoolExecutor Queuing Tasks Parallelizing Loops
Thread Pool Size The number of threads in a thread pool can be optimized for the target tasks Should be sized dynamically int cpucount = Runtime.getRuntime().availableProcessors(); Based on the type of task I/O bound vs. CPU bound Sizing is not an absolute number, but rather too big or too small
How Many Threads? A large thread pool: incurs significant overhead for management, locks-up memory while active or blocked A small thread pool: reduces parallelism impacts throughput
Sizing Thread Pools Consider the nature of the tasks How many resources (sockets, file handles, etc.) are required? What type of task (I/O vs CPU) How many processors are available? How much memory is available? Are other required resources scarce (i.e. JDBC connection) Consider different execution policies (thread pools) for different types of tasks
Thread Pool Sizing Ncpu = number of available CPUs Ucpu = target CPU utilization (0 <= Ucpu <= 1 W/C = ratio of wait time to compute time (blocked) Nthreads = Ncpu * Ucpu * (1 + W/C)
Thread Pools Thread Pool Limitations Sizing Thread Pools ThreadPoolExecutor Queuing Tasks Parallelizing Loops
Executor Default Executors provide many behaviors Executors may be customized after construction ThreadPoolExecutor
Customizing ThreadPoolExecutor Hooks are provides in ThreadPoolExecutor beforeexecute - called before a task is executed afterexecute - called after a task is run, or an exception terminated - called when the thread pool completes shutdown Ideal for logging and statistics gathering http://jcip.net/listings/timingthreadpool.java
Thread Pools Thread Pool Limitations Sizing Thread Pools ThreadPoolExecutor Queuing Tasks Parallelizing Loops
Queued Tasks Thread Pools maintain a queue of pending tasks Tasks may still be scheduled faster than they can execute Resulting in resource consumption and instability LinkedBlockingQueue is the default Executor queue mechanism BlockQueues provide one potential solution An optional parameter to a ThreadPoolExecutor
Bounded Blocking Queues Bounded blocking queues provide a better solution Submitted tasks will be queued to the limit of the queue size Clients will block when attempting to submit tasks past the upper bound Requires a saturation policy Queue size and thread pool size must be considered together
Synchronous Queue A synchronous queue can be used for thread pool task submission Requires large or unbound pool sizes Consumer must be faster than the producer Improves response time to new tasks May invoke the saturation policy sooner
PriorityQueue A priority queue can be used to create a scheduling policy in conjunction with an execution policy
Saturation Policy How does a client respond to a rejected task submission? Use a provided rejection policy handler AbortPolicy - throws RejectedExecutionException CallerRunsPolicy - the client runs the task DiscardPolicy - deletes the submitted task DiscardOldestPolicy - replaces the next task to run(!) Define a custom rejection handler setrejectedexecutionhandler
Block on Full Work Queue http://jcip.net/listings/boundedexecutor.java
Thread Factories Executors create threads through the default thread factory Returns a plain, non-daemon thread Custom ThreadFactories can be specified for Executors
Custom Thread Factory Reasons: Trap uncaught exceptions Provide names for threads Provide debug logging Apply security policies Include performance statistics Threads created Threads destroyed
Thread Pools Thread Pool Limitations Sizing Thread Pools ThreadPoolExecutor Queuing Tasks Parallelizing Loops
Sequential to Parallel
Sequential to Parallel Candidates have: Complex computations in the loop body Potentially blocking I/O in the loop body Independent processing No ordering dependencies Enough loop elements to offset the overhead The loop completes after tasks are queued Not after the processing!
Recursive Parallelism
Awaiting Parallel Results