Introduction to Python programming, II

Transcription

1 Grid Computing Competence Center Introduction to Python programming, II Riccardo Murri Grid Computing Competence Center, Organisch-Chemisches Institut, University of Zurich Nov. 16, 2011

2 Today s class More Python: objects and classes, list comprehensions, iteration and objects (again), exceptions, decorators, callable objects, etc. These slides are available for download from:

3 User-defined objects import unittest as ut class SpTest(ut.TestCase): def test_sp_1(self): (x, y) = sp(100, [5, 75, 25]) self.asserttrue((x,y) in [(75, 25), (25,75)]) def test_sp_2(self): (x,y) = sp(8, [2,1,9,4,4,56,90,3]) self.assertequal((x,y), (4,4)) This was your first encounter with Python object, during the last Lab session. Today we are going to examine Python objects and classes in more detail.

4 Declare a new class, import unittest as ut ihneriting from ut.testcase. The body of class SpTest(ut.TestCase): the declaration is indented relative to the class def test_sp_1(self): statement. (x, y) = sp(100, [5, 75, 25]) self.asserttrue((x,y) in [(75,25), (25,75)]) def test_sp_2(self): (x,y) = sp(8, [2,1,9,4,4,56,90,3]) self.assertequal((x,y), (4,4))

5 A method declaration looks import unittest as ut exactly like a function definition. Every method class SpTest(ut.TestCase): must have at least one argument, named self. def test_sp_1( self ): (x, y) = sp(100, [5, 75, 25]) self.asserttrue((x,y) in [(75,25), (25,75)]) def test_sp_2(self): (x,y) = sp(8, [2,1,9,4,4,56,90,3]) self.assertequal((x,y), (4,4))

6 self is a reference to the import unittest as ut object instance (like, e.g., this in Java). It is used to class SpTest(ut.TestCase): access attributes and invoke methods of the def test_sp_1(self): instance itself. (x, y) = sp(100, [5, 75, 25]) self.asserttrue((x,y) in [(75, 25), (25,75)]) def test_sp_2(self): (x,y) = sp(8, [2,1,9,4,4,56,90,3]) self.assertequal((x,y), (4,4))

7 The self argument Every method of a Python object always has self as first argument. However, you do not specify it when calling a method: it s automatically inserted by Python: >>> class ShowSelf(object):... def show(self):... print(self)... >>> x = ShowSelf() # construct instance >>> x.show() # self automatically inserted! < main.showself object at 0x299e150> The self variable is a reference to the object instance itself. You need to use self when accessing methods or attributes of this instance.

8 import unittest as ut These call methods defined on the current instance. class SpTest(ut.TestCase): Quiz: Where are they defined, exactly? def test_sp_1(self): (x, y) = sp(100, [5, 75, 25]) self.asserttrue ((x,y) in [(75, 25), (25,75)]) def test_sp_2(self): (x,y) = sp(8, [2,1,9,4,4,56,90,3]) self.assertequal ((x,y), (4,4))

9 Name resolution rules Within a function body, names are resolved according to the LEGB rule: L Local scope: any names defined in the current function; E Enclosing function scope: names defined in enclosing functions (outermost last); G global scope: names defined in the toplevel of the current module; B Built-in names (i.e., Python s module). builtins Any name that is not in one of the above scopes must be qualified. So you have to write self.assertequal to call a method on this instance, ut.testcase to mean a class defined in module ut, etc.

10 An easy exercise A dotfile is a file whose name starts with a dot character.. How can you list all dotfiles in a given directory? (Recall that the Python library call for listing the entries in a directory is os.listdir())

11 A very basic solution Use a for loop to accumulate the results into a list: dotfiles = [ ] for entry in os.listdir(path): if entry.startswith(. ): dotfiles.append(entry)

12 List comprehensions, I Python has a better and more compact syntax for filtering elements of a list and/or applying a function to them: dotfiles = [ entry for entry in dotfiles if entry.startswith(. ) ] This is called a list comprehension.

13 List comprehensions, II The general syntax of a list comprehension is: where: [ expr for var in iterable if condition ] expr is any Python expression; iterable is a (generalized) sequence; condition is a boolean expression, depending on var; var is a variable that will be bound in turn to each item in iterable which satisfies condition. The if condition part is optional.

14 Generator expressions List comprehensions are a special case of generator expressions: ( expr for var in iterable if condition ) A generator expression is a valid iterable and can be used to initialize tuples, sets, dicts, etc.: # the set of square numbers < 100 squares = set(n*n for n in range(10)) Generator expressions are valid expression, so they can be nested: # cartesian product of sets A and B C = set( (a,b) for a in A for b in B )

15 Generators Generator expressions are a special case of generators. A generator is like a function, except it uses yield instead of return: def squares(): n = 0 while True: yield n*n n += 1 At each iteration, execution resumes with the statement logically following yield in the generator s execution flow. There can be multiple yield statements in a generator. Reference:

16 The Iterator Protocol An object can function as an iterator iff it implements a next() method, that: either returns the next value in the iteration, or raises StopIteration to signal the end of the iteration. An object can be iterated over with for if it implements a iter () method. Reference:

17 Iterate over the words in class WordIterator(object): the given text: split the text at white spaces, and def init (self, text): return the parts self._words = text.split() one by one. def next(self): if len(self._words) > 0: return self._words.pop(0) else: raise StopIteration def iter (self): return self Source code available at:

18 class WordIterator( object ): def init (self, text): self._words = text.split() def next(self): if len(self._words) > 0: return self._words.pop(0) else: raise StopIteration def iter (self): return self Every class must inherit from a parent class. If there s no other class, inherit from the object class. (Root of the class hierarchy.)

19 class WordIterator(object): The constructor. def init (self, text): self._words = text.split() def next(self): if len(self._words) > 0: return self._words.pop(0) else: raise StopIteration def iter (self): return self

20 Constructors The init method is the object constructor. It should never return any value (other than None). However, you call a constructor by class name: # make w an instance of WordIterator w = WordIterator("some text") (Again, note that the self part is automatically inserted by Python.)

21 No overloading Python does not allow overloading of functions. Any function. Hence, no overloading of constructors. So: a class can have one and only one constructor.

22 Constructor chaining When a class is instanciated, Python only calls the first constructor it can find in the class inheritance call-chain. If you need to call a superclass constructor, you need to do it explicitly: class Application(Task): def init (self,...): # do Application-specific stuff here Task. init (self,...) # some more Application-specific stuff Calling a superclass constructor is optional, and it can happen anywhere in the init method body.

23 Multiple-inheritance Python allows multiple inheritance. Just list all the parent classes: class C(A,B): # class definition With multiple inheritance, it is your responsibility to call all the needed superclass constructors. Python uses the C3 algorithm to determine the call precedence in an inheritance chain. You can always query a class for its method resolution order, via the mro attribute: >>> C. mro (<class ex.c >, <class ex.a >, <class ex.b >, <type object >)

24 class WordIterator(object): This creates an attribute of the current object. def init (self, text): self. words = text.split() def next(self): if len(self._words) > 0: return self._words.pop(0) else: raise StopIteration def iter (self): return self

25 Object attributes A Python object is (in particular) a key-value mapping: attributes (keys) are valid identifiers, values can be any Python object. Any object has attributes, which you can access (create, read, overwrite) using the dot notation: # create or overwrite the name attribute of w w.name = "Joe" # get the value of w.name and print it print (w.name) So, in the constructor you create the required instance attributes using self.var =... Note: also methods are attributes!

26 No access control There are no public / private /etc. qualifiers for object attributes. Any code can create/read/overwrite/delete any attribute on any object. There are conventions, though: protected attributes: name private attributes: name (But again, note that this is not enforced by the system in any way.)

27 class WordIterator(object): def init (self, text): self._words = text.split() def next(self): if len(self._words) > 0: return self._words.pop(0) else: raise StopIteration def iter (self): return self Raise the StopIteration exception if there are no more words to return. (More on exceptions in a moment.)

28 class WordIterator(object): def init (self, text): self._words = text.split() def next(self): if len(self._words) > 0: return self._words.pop(0) else: raise StopIteration def iter (self): return self The iter method should return a valid iterator. Since this object is an iterator, just return self.

29 Using iterators Iterators can be used in a for loop: >>> for word in WordIterator("a nice sunny day"):... print * +word+ *,... *a* *nice* *sunny* *day* They can be composed with other iterators for effect: >>> for n, word in enumerate(worditerator("a...")):... print str(n)+ : +word,... 0:a 1:nice 2:sunny 3:day See also:

30 Exceptions Exceptions are objects that inherit from the built-in Exception class. To create a new exception just make a new class: class NewKindOfError(Exception): """ Do use the docstring to document what this error is about. """ pass Exceptions are handled by class name, so they usually do not need any new methods (although you are free to define some if needed). See also:

31 try: # code that might raise an exception except SomeException: # handle some exception except AnotherException, ex: # the actual Exception instance # is available as variable ex else: # performed on normal exit from try finally: # performed on exit in any case The optional else clause is executed if and when control flows off the end of the try clause. The optional finally clause is executed on exit from the try or except block in any case. Reference: stmts.html#try

32 Raising exceptions Use the raise statement with an Exception instance: if an_error_occurred: raise AnError("Spider sense is tingling.") Within an except clause, you can use raise with no arguments to re-raise the current exception: try: something() except ItDidntWork: do_cleanup() # re-raise exception to caller raise

33 Exception handling example Read lines from a CSV file, ignoring those that do not have the required number of fields. If other errors occur, abort. Close the file when done. job_state = { } # empty dict try: csv_file = open( jobs.csv, r ) for line in csv_file: line = line.strip() # remove trailing newline try: name, jobid, state = line.split(",") except ValueError: continue # ignore line job_state[jobid] = state except IOError: raise # up to caller finally: csv_file.close()

34 A common case The cleanup pattern is so common that Python has a special statement to deal with it: with open( jobs.csv, r ) as csv_file: for line in csv_file: line = line.strip() # remove trailing newline try: name, jobid, state = line.split(",") except ValueError: continue # ignore line job_state[jobid] = state The with statement ensures that the file is closed upon exit from the with block (for whatever reason). Reference: stmts.html#with

35 The context manager protocol Any object can be used in a with statement, provided it defines the following two methods: enter () Called upon entrance of the with block; it return value is assigned to the variable following as (if any). exit (exc_cls, exc_val, exc_tb) Called with three arguments upon exit from the block. If an exception occurred, the three arguments are the exception type, value and traceback; otherwise, the three argument are all set to None Quiz: Can you think of other examples where this could be useful? See also:

36 Functions are first-class in Python: so you can pass them as arguments to other functions, return them from a function, and hence modify them. The decorator def func(...): # function body is an alternate and more readable form for: def func(...): # function body func = decorator(func) Here, decorator is a function that takes a function as argument and returns (another) function.

37 Decorators Decorators are expressions that take a function and evaluate to a function. They are made possible (and interesting) because of two Python features: Functions are first-class. Functions can be defined inside other functions.

38 A debug decorator, I This is a real decorator, that modifies a function to print a log line with its calling parameters. def trace(func): name = func.func_name def substitute(*posargs, **kwargs): print ("Calling %s with args=%r, kwargs=%r" % (name, posargs, kwargs)) return func(*posargs, **kwargs) return substitute Source code available at:

39 A debug decorator, II So, for example this def factorial(n): return 1 if n==0 else n*factorial(n-1) Produces the following output: >>> factorial(3) Calling factorial with args=(2,), kwargs={} Calling factorial with args=(1,), kwargs={} Calling factorial with args=(0,), kwargs={} 6

40 Interlude: the ternary def factorial(n): return 1 if n==0 else n*factorial(n-1) Incidentally, this is the ternary operator, which in C and Java reads condition? if true : if false

41 The debug decorator explained def trace(func): name = func.func_name def substitute(*posargs, **kwargs): print ("Calling %s with args=%r, kwargs=%r" % (name, posargs, kwargs)) return func(*posargs, **kwargs) return substitute A decorator is just an ordinary function...

42 The debug decorator explained def trace(func): name = func.func_name def substitute(*posargs, **kwargs): print ("Calling %s with args=%r, kwargs=%r" % (name, posargs, kwargs)) return func(*posargs, **kwargs) return substitute... that creates an inner function...

43 The debug decorator explained def trace(func): name = func.func_name def substitute(*posargs, **kwargs): print ("Calling %s with args=%r, kwargs=%r" % (name, posargs, kwargs)) return func(*posargs, **kwargs) return substitute... which is then returned as result of the outer function.

44 The debug decorator explained def trace( func ): name = func.func name def substitute(*posargs, **kwargs): print ("Calling %s with args=%r, kwargs=%r" % ( name, posargs, kwargs)) return func (*posargs, **kwargs) return substitute Any names that are bound in the outer function are available in the inner function. Each invocation of the outer function creates a new set of bindings (i.e., two inner functions created by different invocations will not see each other s bindings).

45 star-arguments, I def trace(func): name = func.func_name def substitute( *posargs, **kwargs): print ("Calling %s with args=%r, kwargs=%r" % (name, posargs, kwargs)) return func(*posargs, **kwargs) return substitute In a function definition, * args indicates that the variable args is bound to the tuple of actual parameters that were passed to the function.

46 star-arguments, II def trace(func): name = func.func_name def substitute(*posargs, **kwargs): print ("Calling %s with args=%r, kwargs=%r" % (name, posargs, kwargs)) return func( *posargs, **kwargs) return substitute In a function call, * args indicates that the remaining positional arguments are taken from the sequence args.

47 star-star-arguments def trace(func): name = func.func_name def substitute(*posargs, **kwargs ): print ("Calling %s with args=%r, kwargs=%r" % (name, posargs, kwargs)) return func(*posargs, **kwargs ) return substitute Similarly, ** kwargs in a definition binds variable kwargs to a dictionary containing the passed keyword arguments. In a function call, ** kwargs takes additional keyword arguments from dictionary kwargs.

48 What s a decorator, again? Decorators are functions callable objects that take a function and return a function. An object is callable iff it defines a call () method. Functions are callable objects. class CallMe(object): def call (self, name="baby"): return ("Call me, %s" % name) >>> c = CallMe() >>> c() Call me, baby >>> c("mister") Call me, Mister A callable object may be called as if it were a function. The signature of the call () method determines the arguments used in the call.

49 Example: The memoize decorator The memoize decorator caches function results and re-uses them instead of running the computation again. class memoize(object): def init (self, func): self._func = func self._cache = dict() def call (self, *args): if args not in self._cache: self._cache[args] = self._func(*args) return self._cache[args] Source code available at:

50 Class attributes Classes are Python objects too, hence they can have attributes. Class attributes can be created with the variable assignment syntax in a class definition block: class A(object): class_attr = value def init (self): #... Class attributes are shared among all instances of the same class!

51 Class attributes example class AutoInc(object): _cnt = 0 def init (self): AutoInc._cnt += 1 self._value = AutoInc._cnt def value(self): return self._value Quiz: Create three instances of the class: u1, u2, u3. What would the value() method return for each of them?

52 Further reading Idiomatic Python: A presentation of Python programming idioms for writing better (simpler, cleaner and often faster) code, by David Goodger. The Zen of Python in 3 days: Great set of slides from a 3-day course, covering topics from the very basics of Python programming to advanced real-world topics like SQL ORM usage and web programming. A commented list of online resources on the course Wiki:

53 import antigravity (You need at least Python 2.7 for this to work, though.)