Thread Pools SE 441. Prof. Bullinger

Transcription

1 Thread Pools SE 441 Prof. Bullinger

2 Thread Pools Thread Pool Limitations Sizing Thread Pools ThreadPoolExecutor Queuing Tasks Parallelizing Loops

3 Thread Pool Limitations Task Dependencies Different types of tasks require different execution policies Independent tasks are ideal Dependent tasks induce coupling Timing Results Side-effects Other tasks

4 Thread Confinement Thread confinement couples a task to its execution policy A single-threaded Executor does not require a thread-safe task Changing to a multi-threaded Executor introduces potential thread-safety issues Time-sensitive tasks may be impeded by an execution policy A long task in a single-threaded Executor may affect response time of other tasks

5 ThreadLocal ThreadLocal provides thread-specific state Executors reuse threads, rendering thread-specific state unusable Threads may also come and go, depending on the execution policy of the Executor Limit ThreadLocal to the lifecycle of a Task

6 Thread Pool Heuristics Thread Pools work best when tasks are homogenous and independent Don t mix tasks that depend on each other in the same Executor One task blocks, waiting for the results of the other The other cannot run due to limited resources A single-threaded thread pool A thread-pool with no available threads

7 Thread Pool Heuristics Thread Pools work best when tasks are homogenous and independent Don t mix long-running and short-running tasks Eventually, long-running tasks dominate the thread pool Responsiveness may suffer Suggest adding timeouts to blocking calls to ensure liveness Large or unbounded thread pools minimize risk Document the execution policy requirement for a task!

8 Thread Pools Thread Pool Limitations Sizing Thread Pools ThreadPoolExecutor Queuing Tasks Parallelizing Loops

9 Thread Pool Size The number of threads in a thread pool can be optimized for the target tasks Should be sized dynamically int cpucount = Runtime.getRuntime().availableProcessors(); Based on the type of task I/O bound vs. CPU bound Sizing is not an absolute number, but rather too big or too small

10 How Many Threads? A large thread pool: incurs significant overhead for management, locks-up memory while active or blocked A small thread pool: reduces parallelism impacts throughput

11 Sizing Thread Pools Consider the nature of the tasks How many resources (sockets, file handles, etc.) are required? What type of task (I/O vs CPU) How many processors are available? How much memory is available? Are other required resources scarce (i.e. JDBC connection) Consider different execution policies (thread pools) for different types of tasks

12 Thread Pool Sizing Ncpu = number of available CPUs Ucpu = target CPU utilization (0 <= Ucpu <= 1 W/C = ratio of wait time to compute time (blocked) Nthreads = Ncpu * Ucpu * (1 + W/C)

13 Thread Pools Thread Pool Limitations Sizing Thread Pools ThreadPoolExecutor Queuing Tasks Parallelizing Loops

14 Executor Default Executors provide many behaviors Executors may be customized after construction ThreadPoolExecutor

15 Customizing ThreadPoolExecutor Hooks are provides in ThreadPoolExecutor beforeexecute - called before a task is executed afterexecute - called after a task is run, or an exception terminated - called when the thread pool completes shutdown Ideal for logging and statistics gathering

16 Thread Pools Thread Pool Limitations Sizing Thread Pools ThreadPoolExecutor Queuing Tasks Parallelizing Loops

17 Queued Tasks Thread Pools maintain a queue of pending tasks Tasks may still be scheduled faster than they can execute Resulting in resource consumption and instability LinkedBlockingQueue is the default Executor queue mechanism BlockQueues provide one potential solution An optional parameter to a ThreadPoolExecutor

18 Bounded Blocking Queues Bounded blocking queues provide a better solution Submitted tasks will be queued to the limit of the queue size Clients will block when attempting to submit tasks past the upper bound Requires a saturation policy Queue size and thread pool size must be considered together

19 Synchronous Queue A synchronous queue can be used for thread pool task submission Requires large or unbound pool sizes Consumer must be faster than the producer Improves response time to new tasks May invoke the saturation policy sooner

20 PriorityQueue A priority queue can be used to create a scheduling policy in conjunction with an execution policy

21 Saturation Policy How does a client respond to a rejected task submission? Use a provided rejection policy handler AbortPolicy - throws RejectedExecutionException CallerRunsPolicy - the client runs the task DiscardPolicy - deletes the submitted task DiscardOldestPolicy - replaces the next task to run(!) Define a custom rejection handler setrejectedexecutionhandler

22 Block on Full Work Queue

23 Thread Factories Executors create threads through the default thread factory Returns a plain, non-daemon thread Custom ThreadFactories can be specified for Executors

24 Custom Thread Factory Reasons: Trap uncaught exceptions Provide names for threads Provide debug logging Apply security policies Include performance statistics Threads created Threads destroyed

25 Thread Pools Thread Pool Limitations Sizing Thread Pools ThreadPoolExecutor Queuing Tasks Parallelizing Loops

26 Sequential to Parallel

27 Sequential to Parallel Candidates have: Complex computations in the loop body Potentially blocking I/O in the loop body Independent processing No ordering dependencies Enough loop elements to offset the overhead The loop completes after tasks are queued Not after the processing!

28 Recursive Parallelism

29 Awaiting Parallel Results