Lecture 9. Introduction to Python! Lecture 9

Transcription

1 Lecture 9 Introduction to Python Lecture 9

2 Lecture 9 Summary Threads Daemons Timer Event Reentrant Locks Producer-Consumer problem Condition Semaphores Design Patterns Singleton Façade Factory Proxy Adapter

3 Daemons import threading import time def do_something_non_daemonic(): print("starting non-daemonic...") print("ending non-daemonic...") def do_something_daemonic(): print("starting daemonic...") time.sleep(5) print("ending non-daemonic...") t1 = threading.thread(name = "t1", target = do_something_non_daemonic) t2 = threading.thread(name = "t2", target = do_something_daemonic) t2.setdaemon(true) t1.start() t2.start() Output: starting non-daemonic... ending non-daemonic... starting daemonic... Process finished with exit code 0 Be sure to set the daemon flag for the thread before calling the start() method, otherwise you ll fall in a RuntimeError. It is not possible to alter that flag after the thread was started

4 Daemons A thread can be flagged as daemon thread, having the significance that the main program will exit only when no other threads than daemons are left running. Keep in mind that the daemons threads are abruptly stopped at shutdown. If they are using resources like connections, files, etc, they may not be properly released. Given all these, daemons are useful when, during the main runs, it is ok to kill them after all the other non-daemon threads have completed their work. Some examples of daemon threads are the ones for sending keepalive packets, running periodically garbage collection, and more. The threading.thread class has the following methods dealing with daemon threads: isdaemon() returns True if the current threads was flagged as being a daemon setdaemon() sets the daemon flag for the current thread

5 Enumerate threads import threading import random import time class MyThread(threading.Thread): def init (self): threading.thread. init (self) def run(self): name = threading.current_thread().getname() fun(name) def fun(name): k = random.randint(5, 10) print("sleeping {} seconds...".format(k)) time.sleep(k) print("ending sleep for {}".format(name)) if name == " main ": for i in range(5): t = MyThread(); t.start() main_thread = threading.current_thread() for j in threading.enumerate(): print("thread name {}".format(j.getname())) # we cannot join a thread to itself. Through the # threads returned by enumerate() # one can found the main thread too if j is not main_thread: j.join() Output: sleeping 6 seconds... sleeping 10 seconds... sleeping 8 seconds... sleeping 9 seconds... sleeping 6 seconds... Thread name MainThread Thread name Thread-1 ending sleep for Thread-1 Thread name Thread-2 ending sleep for Thread-5 ending sleep for Thread-3 ending sleep for Thread-4 ending sleep for Thread-2 Thread name Thread-4 Thread name Thread-5 Thread name Thread-3 enumerate() will gather all active threads, daemons or non-daemons, including the main thread. All the threads being in Alive state will be retrieved

6 Timer object import threading def func(time_interval): print("executed after {} seconds...".format(time_interval)) time_interval = 3 # seconds t = threading.timer(time_interval, func, [time_interval]) t.start() Executed after 3 seconds... Output When we need to execute a piece of code after a specific time interval, one can use the threading.timer object. Timer is a subclass of threading.thread class. The timer can be interrupted inside the interval by calling the cancel() method. Notice how we pass the arguments to the callback function func, in the threading.timer creation code

7 Events def wait_for_event(e): print("starting wait_for_event...") event_set = e.wait() print("in wait_for_event event_set is {}".format(event_set)) def wait_for_event_timeout(e, timeout): while not e.isset(): print("starting wait_for_event_timeout...") event_set = e.wait(timeout) print("in wait_for_event_timeout event_set is {}".format(event_set)) if event_set: print("processing event...") else: print("do something else...") if name == " main ": e = threading.event() t1 = threading.thread(name='blocking', target=wait_for_event, args=(e,)) t1.start() t2 = threading.thread(name='non_blocking', target=wait_for_event_timeout, args=(e, 3)) t2.start() print("before generating event...") time.sleep(3) e.set() print("after generating event...") One of the simplest mechanism of communication among threads is the one using the Event object. In this scenario, a thread signals an event, another thread is waiting for that event to occur. Internally, the Event object has a flag which is set using the set() method, or cleared using clear() method. The wait() method blocks the thread until the Event flag is set to true starting wait_for_event... starting wait_for_event_timeout... before generating event... in wait_for_event_timeout event_set is False do something else... starting wait_for_event_timeout... after generating event... in wait_for_event_timeout event_set is True processing event... in wait_for_event event_set is True

8 Reentrant locks import threading lock = threading.lock() print("acquiring lock first time {}".format(lock.acquire())) print("acquiring lock second time {}".format(lock.acquire(false))) print("this is displayed if the main thread is not blocked") Output: Acquiring lock first time True Acquiring lock second time False This is displayed if the main thread is not blocked A Lock object can be acquired only once, even if this is performed from the same thread. One can see from this example that the second attempt to acquire the lock will fail, the acquire() function is returning False. The call to the acquire() method was made by changing the default argument of the blocking parameter to False, meaning that the thread won t block until the lock will be released, but returning immediately with a result of False.

9 Reentrant locks import threading lock = threading.rlock() print("acquiring lock first time {}".format(lock.acquire())) print("acquiring lock second time {}".format(lock.acquire())) print("this is displayed if the main thread is not blocked") Output: Acquiring lock first time True Acquiring lock second time True This is displayed if the main thread is not blocked Changing the Lock object with a re-entrant lock object RLock will solve the problem of acquiring the lock multiple times by a thread. One can see that the result of acquire() method is True, which means that the lock was successfully acquired second time also. RLock object is useful in recursive functions where the code is executed multiple times.

10 The Producer-Consumer problem The producer-consumer problem is a well-known example of synchronization among multiple processes. It describes two distinct processes, one called producer and one called consumer, which share a common buffer, having a fixed size, used as a queue. The producer generates data and push it into the buffer, while the consumer consumes the data stored in the buffer, one piece at a time. One have to make sure that the producer won t try to add data in the buffer while it is full and the consumer won t try to remove data if the buffer is empty. The processes run simultaneously, so we have to synchronize them.

11 The Condition object A Condition object is always associated with a kind of lock. The condition object can be passed in or it can be created by default. The Condition object allows one or more threads to wait until notified by another thread. Being associated to a lock, it has acquire() and release() methods, no need for explicit Lock object. In the case of producer-consumer problem, the consumer needs to wait for a notification, using the instance of a Condition object, while the producer needs to notify the consumer using the same instance of Condition object. This means that the Condition instance will be passed between producer and consumer. On the next slide, a very simplified example of producer-consumer is explained.

12 The Condition object import threading import time def consumer(cond): print("starting the consumer thread {}...".format(threading.current_thread().getname())) with cond: print("consumer thread {} is waiting...".format(threading.current_thread().getname())) cond.wait() print("consumer thread {} consumed the resource...".format(threading.current_thread().getname())) def producer(cond): print("starting the producer thread {}...".format(threading.current_thread().getname())) with cond: print("producing a resource...") print("inform all the consumers about it...") cond.notifyall() if name == " main ": condition = threading.condition() consumer_one = threading.thread(name="consumer-one", target=consumer, args=(condition,)) consumer_two = threading.thread(name="consumer-two", target=consumer, args=(condition,)) producer_one = threading.thread(name="producer-one", target=producer, args=(condition,)) consumer_one.start() time.sleep(3) consumer_two.start() time.sleep(3) producer_one.start() Starting the consumer thread Consumer-One... Consumer thread Consumer-One is waiting... Starting the consumer thread Consumer-Two... Consumer thread Consumer-Two is waiting... Starting the producer thread Producer-One... Producing a resource... Inform all the consumers about it... Consumer thread Consumer-One consumed the resource... Consumer thread Consumer-Two consumed the resource...

13 Semaphores Semaphores are some of the oldest synchronization mechanisms. They were invented by Edsger W. Djikstra in 1962 or A Semaphore manages an internal counter which is decremented by each acquire() method and incremented by each release(). The counter can never go below zero. When acquire() finds that the counter reached zero, it block until another thread will call a release(). In contrast to the mutexes (or locks), semaphores allow many threads to run concurrently and access a resource, but in the same time they limit the overall number of accesses. Semaphores are useful when we need to support concurrent downloads, or concurrent connections to a web server or to a database, but also one can find other use cases, where concurrency is required.

14 Semaphores import threading import time class ThreadPool(): def init (self): self.active = [] self.lock = threading.lock() def makeactive(self, name): with self.lock: self.active.append(name) print("running {}".format(self.active)) def makeinactive(self, name): with self.lock: self.active.remove(name) print("running {}".format(self.active)) def func(s, pool): print("waiting to join the threading pool...") with s: name = threading.current_thread().getname() pool.makeactive(name) time.sleep(1) pool.makeinactive(name) if name == " main ": pool = ThreadPool() semaphore = threading.semaphore(5) for i in range(10): t = threading.thread(name="thread-" + str(i), target=func, args=(semaphore, pool)) t.start()

15 Semaphores The output from the previous slide Waiting to join the threading pool... Running ['Thread-0'] Waiting to join the threading pool... Running ['Thread-0', 'Thread-1'] Waiting to join the threading pool... Running ['Thread-0', 'Thread-1', 'Thread-2'] Waiting to join the threading pool... Running ['Thread-0', 'Thread-1', 'Thread-2', 'Thread-3'] Waiting to join the threading pool... Running ['Thread-0', 'Thread-1', 'Thread-2', 'Thread-3', 'Thread-4'] Waiting to join the threading pool... Waiting to join the threading pool... Waiting to join the threading pool... Waiting to join the threading pool... Waiting to join the threading pool... Running ['Thread-1', 'Thread-2', 'Thread-3', 'Thread-4'] Running ['Thread-2', 'Thread-3', 'Thread-4'] Running ['Thread-3', 'Thread-4'] Running ['Thread-3', 'Thread-4', 'Thread-5'] Running ['Thread-4', 'Thread-5'] Running ['Thread-5'] Running ['Thread-5', 'Thread-6'] Running ['Thread-5', 'Thread-6', 'Thread-7'] Running ['Thread-5', 'Thread-6', 'Thread-7', 'Thread-8'] Running ['Thread-5', 'Thread-6', 'Thread-7', 'Thread-8', 'Thread-9'] Running ['Thread-5', 'Thread-7', 'Thread-8', 'Thread-9'] Running ['Thread-7', 'Thread-8', 'Thread-9'] Running ['Thread-7', 'Thread-8'] Running ['Thread-7'] Running []

16 Design Patterns DESIGN PATTERNS

17 Singleton Design Pattern Singleton Design Pattern (aka Highlander) There can be only one In software engineering, the singleton pattern is a design pattern that restricts the instantiation of a class to one object. This is useful when exactly one object is needed to coordinate actions across the system. The concept is sometimes generalized to systems that operate more efficiently when only one object exists, or that restrict the instantiation to a certain number of objects. The term comes from the mathematical concept of a singleton. (Wikipedia definition) The most popular use of the singleton is where we have objects that deal with things like controllers in MVC design pattern or database connections. Sometimes, resource creation can be very costly, so we limit it to a single instance that will be used again and again. The Singleton design pattern is a way that assures that a class will have one and only one instance.

18 Java Singleton version public class Singleton { private static Singleton instance = null; private Singleton(){ } Output: public static Singleton getinstance(){ if(instance == null){ instance = new Singleton(); } return instance; } public static void main(string[] args){ Singleton s1 = Singleton.getInstance(); Singleton s2 = Singleton.getInstance(); } } System.out.println(s1.hashCode()); System.out.println(s2.hashCode());

19 Python Singleton version Version 1 class Singleton(): def new (cls, *args, **kwargs): if not hasattr(cls, '_inst'): cls._inst = super(singleton, cls). new (cls, *args, **kwargs) return cls._inst s1 = Singleton() print(id(s1)) s2 = Singleton() print(id(s2)) s3 = Singleton() print(id(s3)) Output:

20 Python Singleton Version Version 2 class Singleton(): instance = None def new (cls, *args, **kwargs): if Singleton.instance is None: Singleton.instance = object. new (cls, *args, **kwargs) return Singleton.instance def str (self): return self. class. name a = Singleton() print(id(a)) b = Singleton() print(id(b)) Output:

21 Façade Design Pattern Façade Design Pattern The façade pattern is a software engineering design pattern commonly used with Object-oriented programming. (The name is by analogy to an architectural facade.). A facade is an object that provides a simplified interface to a larger body of code, such as a class library. (Wikipedia definition) The façade pattern is a Structural Design Pattern, used to hide the complexity of a system, acting like a simplified interface to the client. This way, the client can easily deal with that interface, avoiding the complicated operation beneath.

22 Façade Design Pattern

23 Façade Design Pattern class VirtualCpu(object): def jump(self): print("{} jump to instruction".format(self. class. name )) def execute(self): print("{} executed the instruction".format(self. class. name )) class VirtualMemory(): def load(self): print("{} loaded the boot address in memory".format(self. class. name )) class VirtualHardDisk(): def read(self): print("{} read the boot sector from disk".format(self. class. name )) class VirtualMachine(): def start(self): virtual_cpu = VirtualCpu() virtual_memory = VirtualMemory() virtual_hard_disk = VirtualHardDisk() virtual_memory.load() virtual_hard_disk.read() virtual_cpu.jump() virtual_cpu.execute() virtual_machine = VirtualMachine() virtual_machine.start() VirtualMemory loaded the boot address in memory VirtualHardDisk read the boot sector from disk VirtualCpu jump to instruction VirtualCpu executed the instruction

24 Factory Design Pattern Factory Design Pattern The factory pattern is a creational design pattern used in software development to encapsulate the processes involved in the creation of objects. (Wikipedia definition) In Factory design pattern we create objects in a way that avoids exposing the creation logic to the client. The newly created object is referred through a common interface. The common use cases for Factory design pattern are: When a class doesn t know what kind of objects will have to create When you have to create an object of anyone of subclasses, depending on the data provided

25 Factory Design Pattern

26 Factory Design Pattern class Soldier: def init (self, name): self.name = name def get_name(self): return self.name def str (self): return self. class. name class Swordsman(Soldier): def init (self, name): print("hi, I am {} and I am a {}".format(name, self. str ())) class Bowman(Soldier): def init (self, name): print("hi, I am {} and I am a {}".format(name, self. str ())) class Factory: def getsoldier(self, name, weapon): if weapon == 'BOW': return Bowman(name) elif weapon == 'SWORD': return Swordsman(name) if name == ' main ': s = Factory().getSoldier('John', 'SWORD') s = Factory().getSoldier('Richard', 'BOW') Hi, I am John and I am a Swordsman Hi, I am Richard and I am a Bowman

27 Proxy Design Pattern Proxy Design Pattern (Surrogate) A proxy, in its most general form, is a class functioning as an interface to something else. The proxy could interface to anything: a network connection, a large object in memory, a file, or some other resource that is expensive or impossible to duplicate. A well-known example of the proxy pattern is a reference counting pointer object. (Wikipedia definition) A Proxy pattern have three distinct and essential elements: The Real Subject (the real class performing all the business, but whose creation is costly) The Proxy Class, acting like an interface to user requests, shielding the Real Subject The Client, making the requests

28 Proxy Design Pattern Typical situation in which a Proxy pattern is recommended: Creation of the Real Subject object is costly in terms of resources, but creation of a Proxy is a cheaper option The Real Subject must be protected from direct use by its clients Sometimes one need to delay the creation of the Real Subject to the point it is really needed, using lazy initialization

29 Proxy Design Pattern

30 Proxy Design Pattern class Image(object): def display_image(self): raise NotImplementedError class RealImage(Image): def init (self, filename): self.filename = filename self.load_image_from_disk() def load_image_from_disk(self): print("loading {}".format(self.filename)) def display_image(self): print("displaying {}".format(self.filename)) class ProxyImage(Image): def init (self, filename): self.filename = filename self.real_image = None def display_image(self): if self.real_image is None: self.real_image = RealImage(self.filename) self.real_image.display_image() image1 = ProxyImage("photo_one") image2 = ProxyImage("photo_two") image1.display_image() image1.display_image() image2.display_image() image2.display_image() image1.display_image() Loading photo_one Displaying photo_one Displaying photo_one Loading photo_two Displaying photo_two Displaying photo_two Displaying photo_one

31 Adapter Design Pattern Adapter Design Pattern The Adapter pattern is a software design pattern (also known as Wrapper, an alternative naming shared with the Decorator pattern) that allows the interface of an existing class to be used as another interface. It is often used to make existing classes work with others without modifying their source code. (Wikipedia definition) The Adapter makes possible to interconnect two different interfaces, making them work together.

32 Adapter Design Pattern

33 Adapter Design Pattern class Soldier(object): def who_am_i(self): raise NotImplementedError def fight(self): raise NotImplementedError class Swordsman(Soldier): def who_am_i(self): print("i am a swordsman") def fight(self): print("i am using a sword") class Cannon(object): def make_boom(self): print("boom") class SoldierAdapter(object): def init (self, cannon): self.cannon = cannon def fight(self): self.cannon.make_boom() if name == " main ": swordsman = Swordsman() swordsman.fight() cannon = Cannon() soldier_adapter = SoldierAdapter(cannon) soldier_adapter.fight() Output: I am using a sword BOOM