Design Patterns & Concurrency. Sebastian Graf, Oliver Haase

Transcription

1 Design Patterns & Concurrency Sebastian Graf, Oliver Haase 1

2 Fundamentals 2

3 Recap Threadsafety What is threadsafety? How can threadsafety be ensured? Amdahls Law 3

4 Recap Threadsafety public final class HitCounter { /** internal state counter variable */ private int counter = 0; /** * Getting the counter right now the counts */ public final int getcounter() { return this.counter; /** * Counting multiple values countstoadd numbers to add to the counter */ public final void increment(final int countstoadd) { counter = counter + countstoadd; /** Counting by one */ public final void incrementbyone() { increment(1); 4

5 Recap Threadsafety public final class SynchronizedCounter { /** internal state counter variable */ private int counter = 0; /** * Getting the counter right now the counts */ public synchronized final int getcounter() { return this.counter; /** * Counting multiple values countstoadd numbers to add to the counter */ public synchronized final void increment(final int countstoadd) { counter = counter + countstoadd; /** Counting by one */ public final void incrementbyone() { increment(1); 5

6 Recap Threadsafety import java.util.list; public final class ListSum { /** * Summing multiple variables * multiple * variables the sum of the variables */ public final int sum(final List<Integer> factors) { int sum = 0; for (final int number : factors) { sum = sum + number; return sum; 6

7 Recap Threadsafety public final class IDHandler { /**Local set of ids*/ private String[] ids = { "e634g", "6fhg4", "hd4fg", "dgf53" ; /**Getting the ids. * the ids */ public String[] getids() { return ids; 7

8 Recap Threadsafety public final class IDHandler { /**Local set of ids*/ private final List<String >ids = new ArrayList()<String>; public IDHandler() { ids.add("e634g"); ids.add("6fhg4"); ids.add("hd4fg"); ids.add("dgf53"); /**Getting the ids. * the ids */ public String[] getids() { return ids.toarray(new String[4]); 8

9 Threadsafe? public class ReadyStater { private static boolean ready; private static int number; private static class ReaderThread extends Thread { public void run() { while (!ready) Thread.yield(); System.out.println(number); public static void main(string[] args) { new ReaderThread().start(); number = 42; ready = true; 9

10 Locking & Visibility 10

11 Difference regarding public class MutableInteger { private int value; public int get() { return value; public void set(int value) { this.value = public class MutableInteger { private long value; public long get() { return value; public void set(long value) { this.value = value; 11

12 Always Synchronize? public class MutableInteger { private long value; public synchronized long get() { return value; public synchronized void set(long value) { this.value = value; public class MutableInteger { private volatile long value; public long get() { return value; public void set(long value) { this.value = value; 12

13 Volatile VS Synchronize No Locking Volatile no synchronization visibility Usage: using if only one thread updates value OR value is not participating to invariants OR any other reason Locking Synchonize synchronization visibility Usage: guarding multiple variables (invariants) accessing within multiple threads 13

14 Publishing and Escaping Publishing: Making a variable visible out of its current scope Escaping: Doing the same by accident public final class IDHandler { /**Local set of ids*/ private String[] ids = { "e634g", "6fhg4", "hd4fg", "dgf53" ; /**Getting the ids. * the ids */ public String[] getids() { return ids; 14

15 Escaping in Construtor public class ThisEscape { public ThisEscape(EventSource source) { source.registerlistener( new EventListener() { public void onevent(event e) { dosomething(e); ); 15

16 Safe Construction Process Do not allow to escape this during construction. public class SafeListener { private final EventListener listener; private SafeListener() { listener = new EventListener() { public void onevent(event e) { dosomething(e); ; public static SafeListener newinstance(eventsource source) { SafeListener safe = new SafeListener(); source.registerlistener(safe.listener); return safe; 16

17 Further techniques to ensure thread safety Confinement Immutability Publication Design Documentation 17

18 Confinement Thread Confinement Binding an object to exactly one thread (which cannot be enforced by the JVM) Ad-hoc Confinement Responsibility of maintaining thread safety is on implementation side (which is mostly fragile) Stack Confinement Object bound to local variables only (happening in methods) ThreadLocal private static ThreadLocal<Connection> connectionholder = new ThreadLocal<Connection>() { public Connection initialvalue() { return DriverManager.getConnection(DB_URL); ; public static Connection getconnection() { return connectionholder.get(); 18

19 Immutability Criterias for Immutability State cannot be modified after construction All fields are final properly constructed Immutable Objects are always thread-safe! public final class ThreeStooges { private final Set<String> stooges = new HashSet<String>(); public ThreeStooges() { stooges.add("moe"); stooges.add("larry"); stooges.add("curly"); public boolean isstooge(string name) { return stooges.contains(name); 19

20 Immutability and class OneValueCache { private final BigInteger lastnumber; private final BigInteger[] lastfactors; public OneValueCache(BigInteger i, BigInteger[] factors) { lastnumber = i; lastfactors = Arrays.copyOf(factors, factors.length); public BigInteger[] getfactors(biginteger i) { if (lastnumber == null!lastnumber.equals(i)) return null; else return Arrays.copyOf(lastFactors, public class VolatileCachedFactorizer implements Servlet { private volatile OneValueCache cache = new OneValueCache(null, null); public void service(servletrequest req, ServletResponse resp) { BigInteger i = extractfromrequest(req); BigInteger[] factors = cache.getfactors(i); if (factors == null) { factors = factor(i); cache = new OneValueCache(i, factors); encodeintoresponse(resp, factors); 20

21 Publishing of data public Holder holder; public void initialize() { holder = new Holder(42); public class Holder { private int n; public Holder(int n) { this.n = n; public void assertsanity() { if (n!= n) throw new AssertionError("This statement is false."); 21

22 Safe publication idioms Initialize objects references from a static initializer Storing a reference into a volatile or AtomicReference Storing a reference into a final field of a proper constructed object Guard the field that stores the object with a lock Publication requirements Immutable objects: can be published with any mechanism Effectively immutable objects: must be published safely Mutable objects: must be published safely and be guared by a lock 22

23 Useful policies for sharing objects Thread-confined: Object is owned exclusively by one thread Shared read-only: Object can be accessed concurrently by multiple threads without additional synchronization Shared thread-safe: Synchronization on object is handled internally. Guarded: Access to the object only possible when given lock held including encluding other thread-safe objects 23

24 Designing Objects Identify the variables that form the object's state; Identify the invariants that constrain the state variables; Establish a policy for managing concurrent access to the object's state 24

25 States? State of an object includes state of all fields primitive: simple n-primitives: n-tuple of the fields LinkedList: all linked nodes Stay aware of pre-/postconditions and invariants: Synchronization-Policy 25

26 Thread-Safety redefined Making a class thread-safe means ensuring that its invariants hold under concurrent access; this requires reasoning about its public final class Counter private long value = 0; public synchronized long getvalue() { return value; public synchronized long increment() { if (value == Long.MAX_VALUE) throw new IllegalStateException("counter overflow"); return ++value; 26

27 Instance Confinement Useful for debugging issues Guarding the access with wrapper Class itself is thread-safe (as are the states only accessible within this class) Example: Collections.synchronizedList representing Decorator-Pattern for guarding collections Caution: wrapped object must not escape! 27

28 Monitor-Based Locking Each object holds its own public final class Counter private long value = 0; public synchronized long getvalue() { return value; public synchronized long increment() { if (value == Long.MAX_VALUE) throw new IllegalStateException("counter overflow"); return ++value; 28

29 Public VS Private Locking public class PrivateLock { private final Object mylock = new Widget widget; void somemethod() { synchronized(mylock) { // Access or modify the state of widget public class PublicLock Widget widget; void synchronized somemethod() { // Access or modify the state of widget 29

30 Delegation of Thread safety public class MutablePoint { public int x, y; public MutablePoint() { x = 0; y = 0; public MutablePoint(MutablePoint p) { this.x = p.x; this.y = p.y; 30

31 Delegation of Thread public class MonitorVehicleTracker private final Map<String, MutablePoint> locations; public MonitorVehicleTracker( Map<String, MutablePoint> locations) { this.locations = deepcopy(locations); public synchronized Map<String, MutablePoint> getlocations() { return deepcopy(locations); public synchronized MutablePoint getlocation(string id) { MutablePoint loc = locations.get(id); return loc == null? null : new MutablePoint(loc); public synchronized void setlocation(string id, int x, int y) { MutablePoint loc = locations.get(id); if (loc == null) throw new IllegalArgumentException("No such ID: " + id); loc.x = x; loc.y = y; private static Map<String, MutablePoint> deepcopy( Map<String, MutablePoint> m) { Map<String, MutablePoint> result = new HashMap<String, MutablePoint>(); for (String id : m.keyset()) result.put(id, new MutablePoint(m.get(id))); return Collections.unmodifiableMap(result); 31

32 public class Point { public final int x, y; public Point(int x, int y) { this.x = x; this.y = y; 32

33 public class DelegatingVehicleTracker { private final ConcurrentMap<String, Point> locations; private final Map<String, Point> unmodifiablemap; public DelegatingVehicleTracker(Map<String, Point> points) { locations = new ConcurrentHashMap<String, Point>(points); unmodifiablemap = Collections.unmodifiableMap(locations); public Map<String, Point> getlocations() { return unmodifiablemap; public Point getlocation(string id) { return locations.get(id); public void setlocation(string id, int x, int y) { if (locations.replace(id, new Point(x, y)) == null) throw new IllegalArgumentException( "invalid vehicle name: " + id); 33

34 Independence of state variables public class VisualComponent { private final List<KeyListener> keylisteners = new CopyOnWriteArrayList<KeyListener>(); private final List<MouseListener> mouselisteners = new CopyOnWriteArrayList<MouseListener>(); public void addkeylistener(keylistener listener) { keylisteners.add(listener); public void addmouselistener(mouselistener listener) { mouselisteners.add(listener); public void removekeylistener(keylistener listener) { keylisteners.remove(listener); public void removemouselistener(mouselistener listener) { mouselisteners.remove(listener); public class NumberRange { // INVARIANT: lower <= upper private final AtomicInteger lower = new AtomicInteger (0); private final AtomicInteger upper = new AtomicInteger (0); public void setlower(int i) { if (i > upper.get()) throw new IllegalArgumentException( "can't set lower to " + i + " > upper"); lower.set(i); public void setupper(int i) { if (i < lower.get()) throw new IllegalArgumentException( "can't set upper to " + i + " < lower"); upper.set(i); public boolean isinrange(int i) { return (i >= lower.get() && i <= upper.get()); 34

35 Extending existing sources Extending? Inheritence? Composition? Locking is based on functionality to be offered Alien-Calls in foreign source code complicate things further. 35

36 What to lock? public class ListHelper<E> { public List<E> list = Collections.synchronizedList(new ArrayList<E>());... public synchronized boolean putifabsent(e x) { boolean absent =!list.contains(x); if (absent) list.add(x); return absent; public class ListHelper<E> { public List<E> list = Collections.synchronizedList(new ArrayList<E>());... public boolean putifabsent(e x) { synchronized (list) { boolean absent =!list.contains(x); if (absent) list.add(x); return absent; 36

37 or public class ImprovedList<T> implements List<T> { private final List<T> list; public ImprovedList(List<T> list) { this.list = list; public synchronized boolean putifabsent(t x) { boolean contains = list.contains(x); if (contains) list.add(x); return!contains; public synchronized void clear() { list.clear(); //... similarly delegate other List methods 37

38 Further Example: BlockingQueues BlockingQueue Pill Item Item Item Item Item Item Item Item Producer Consumer Void call() Void call() 38

39 Documentation Each use of volatile / synchronize reflects a synchronization strategy for mainting invariants/pre-/post-conditions JavaDoc (often not really usable regarding thread safety) Assumptions Annotations 39

40 @NotThreadSafe Field- this (Class.)fieldName methodname() ) 40

41 Outcome of first part It's the mutable state, stupid. Make fields final unless they need to be mutable. Immutable objects are automatically thread-safe. Encapsulation makes it practical to manage the complexity. Guard each mutable variable with a lock. Guard all variables in an invariant with the same lock. Hold locks for the duration of compound actions. A program that accesses a mutable variable from multiple threads without synchronization is a broken program. Don't rely on clever reasoning about why you don't need to synchronize. Include thread safety in the design processor explicitly document that your class is not threadsafe. Document your synchronization policy. 41

42 Structuring Applications 42

43 Task Execution class ThreadPerTaskWebServer { public static void main(string[] args) throws IOException { ServerSocket socket = new ServerSocket(80); while (true) { final Socket connection = socket.accept(); Runnable task = new Runnable() { public void run() { handlerequest(connection); ; new Thread(task).start(); 43

44 How to start a thread Thread 0 void sum(final int[][] numbers) Starting of threads should be based on defined pool of threads Runnable 1 Runnable 2 run() run() Support of lifecycle as well Runnable 3 Runnable 4 run() run() Runnable 5 Runnable 6 run() run() 44

45 A summing example public void sum(final int[][] numbers) { final List<Future<Integer>> futures = new ArrayList<Future<Integer>>(); final ExecutorService exec = Executors.newCachedThreadPool(); for (int i = 0; i < numbers.length; i++) { final int index = i; Callable<Integer> summer = new Callable<Integer>() { public Integer call() throws Exception { return sum; ; futures.add(exec.submit(summer)); // getting all futures for (int i = 0; i < futures.size(); i++) { try { returnval[i] = futures.get(i).get(); catch (final InterruptedException e) { e.printstacktrace(); returnval[i] = Integer.MIN_VALUE; catch (final ExecutionException e) { e.printstacktrace(); returnval[i] = Integer.MIN_VALUE; Thread 0 int[] sum(final int[][] numbers) ExecutorService.submit Callable 1 Callable 2 Integer call() Integer call() FutureList.add FutureList.get Future.get 45

46 Advantages of bound execution Defined Thread lifecycle Adaptive and independent resource management Stability 46

47 Hazards while not using Executors Thread 0 void sum(final int[][] numbers) Runnable 1 Runnable 2 run() run() Runnable 3 Runnable 4 run() run() Runnable 5 Runnable 6 run() run() 47

48 Further Hazards Poor Exception-Handling Overwhelming the resources Hard support for individual multithreaded tasks Unequal tasks Scheduling Cancelation No lifecycle for execution itself 48

49 Executors Fixed Pool Size Queue ExecutorService ThreadFactory Thread Thread Thread Thread Thread Pool Size = 1 Queue ExecutorService ThreadFactory Thread Additional Executors: ScheduledExecutor, CompletionServiceExecutor No Pool Size Queue ExecutorService ThreadFactory Thread Thread Thread Thread Thread Thread Thread Thread Thread 49

50 Stopping Threads... 50

51 There is no safe way to stop a thread public class PrimeGenerator implements Runnable private final List<BigInteger> primes = new ArrayList<BigInteger>(); private volatile boolean cancelled; public void run() { BigInteger p = BigInteger.ONE; while (!cancelled ) { p = p.nextprobableprime(); synchronized (this) { primes.add(p); public void cancel() { Thread.stop(); public synchronized List<BigInteger> get() { return new ArrayList<BigInteger>(primes); 51

52 public class PrimeGenerator implements Runnable private final List<BigInteger> primes = new ArrayList<BigInteger>(); private volatile boolean cancelled; public void run() { BigInteger p = BigInteger.ONE; while (!cancelled ) { p = p.nextprobableprime(); synchronized (this) { primes.add(p); public void cancel() { cancelled = true; public synchronized List<BigInteger> get() { return new ArrayList<BigInteger>(primes); 52

53 Producer / Consumer? class BrokenPrimeProducer extends Thread { private final BlockingQueue<BigInteger> queue; private volatile boolean cancelled = false; BrokenPrimeProducer(BlockingQueue<BigInteger> queue) { this.queue = queue; public void run() { try { BigInteger p = BigInteger.ONE; while (!cancelled) queue.put(p = p.nextprobableprime()); catch (InterruptedException consumed) { public void cancel() { cancelled = true; void consumeprimes() throws InterruptedException { BlockingQueue<BigInteger> primes =...; BrokenPrimeProducer producer = new BrokenPrimeProducer(primes); producer.start(); try { while (needmoreprimes()) consume(primes.take()); finally { producer.cancel(); 53

54 Interruption? class PrimeProducer extends Thread { private final BlockingQueue<BigInteger> queue; PrimeProducer(BlockingQueue<BigInteger> queue) { this.queue = queue; public void run() { try { BigInteger p = BigInteger.ONE; while (!Thread.currentThread().isInterrupted()) queue.put(p = p.nextprobableprime()); catch (InterruptedException consumed) { /* Allow thread to exit */ public void cancel() { interrupt(); 54

55 Interruption Necessary Interruption Policy (what should happen if thread gets interrupted) freeing connection, resources unblocking keeping invariants cleaning up Tasks own not the thread (with the usage of thread pools), policy mostly based on get out of the way 55

56 Interruption What to do if InterruptedException is catched? Propagate the Interruption Restore the internal state public Task getnexttask(blockingqueue<taskgt; queue) { boolean interrupted = false; try { while (true) { try { return queue.take(); catch (InterruptedException e) { interrupted = true; // fall through and retry finally { if (interrupted) Thread.currentThread().interrupt(); 56

57 Interruption of other threads private static final ScheduledExecutorService cancelexec =...; public static void timedrun(runnable r, long timeout, TimeUnit unit) { final Thread taskthread = Thread.currentThread(); cancelexec.schedule(new Runnable() { public void run() { taskthread.interrupt();, timeout, unit); r.run(); 57

58 Future.cancel public static void timedrun(runnable r, long timeout, TimeUnit unit) throws InterruptedException { Future<?> task = taskexec.submit(r); try { task.get(timeout, unit); catch (TimeoutException e) { // task will be cancelled below catch (ExecutionException e) { // exception thrown in task; rethrow throw launderthrowable(e.getcause()); finally { // Harmless if task already completed task.cancel(true); // interrupt if running 58

59 Producer / Consumer, Poison Pill BlockingQueue Pill Item Item Item Item Item Item Item Item Producer Consumer Void call() Void call() 59

60 Executor Shutdown? public class LogService { private final ExecutorService exec = newsinglethreadexecutor();... public void start() { public void stop() throws InterruptedException { try { exec.shutdown(); exec.awaittermination(timeout, UNIT); finally { writer.close(); public void log(string msg) { try { exec.execute(new WriteTask(msg)); catch (RejectedExecutionException ignored) { 60

61 Exceptions? public void log(string msg) throws InterruptedException { if (!shutdownrequested) queue.put(msg); else throw new IllegalStateException("logger is shut down"); 61

62 Always be aware of the exceptions Proactive reaction regarding exceptions Propagate the Interruption Restore the internal state UncaughtExceptionHandler for detection of uncaught exceptions 62

63 Other artifacts Finalizers Daemon Threads JVM Shutdowns 63

64 The last one public interface Runnable2 { public void run() throws Exception; 64