Introduction to Python! Lecture 3

Transcription

1 Introduction to Python Lecture 3

2 Summary Summary Dictionaries Lists comprehensions Dictionaries comprehensions Files Comparisons Lists repetition

3 0 Arrays Characteristics: contiguous memory zones arrays hold elements of the same type Advantages: access elements in a single step, based on index [O(1)] Disadvantages: the size of the array is established from the beginning no guarantee that the memory zone after the last element in the array will be available to extend the array, so growing an array is difficult

4 Linked lists Linked lists Characteristics: non-contiguous memory zones unlimited number of elements Advantages: unlimited number of elements growing a list is dynamically and easy Disadvantages: accessing an element involves iterating the list [O(n) in the worst case, when the desired element is the last in list]

5 Hash tables Big Picture Can we store and access data in one single step, no matter how many elements we have? The answer is YES, using hash tables. Data Hash Function Hashtable

6 Hash tables A hash table is basically an array coupled with a hash function Hash Function 27 The hash function takes a piece of data as input, called a key, and computes a result called a hash value. hash(16) = 0 hash(11) = 3 hash(27) = 3 hash(19) = 3 hash(22) = 6 hash(6) = 6 Key collision hash(key) = hash_value To avoid key collision, we need a strong hash function that produces unique hash values

7 Hash tables Accessing data in a hash table Hash Function 27 Supposing we need to access the element 27 in the hash table, we have to do the following operations: 1. compute hash(27) which is 3, in our case 2. we noticed that at that index there is a linked list, due to some key collisions 3. search in list the element 27 Conclusion: the best case, accessing an element in hash O(1) worst case, accessing an element in hash O(n) insertion an element in hash O(1) deletion an element in hash, between O(1) and O(n)

8 Dictionaries Dictionaries: the most flexible built-in data type in Python collection of unordered elements elements are stored and fetched based on keys instead of off-sets dictionaries have variable length heterogenous, can contain all kind of objects they are arbitrarily nested mutable-mapping (can be change in place, based on key, not on index)

9 Creating a dictionary Creating a dictionary music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973} print(music) print(type(music)) {'year': 1973, 'album': 'Dark Side of the Moon', 'band': 'Pink Floyd } <class 'dict'> Note that the internal order in the dictionary is NOT necessarily the same as the defined one. Python performs some optimisations in order to support a fast key lookup. This is why some operations assuming a fixed left-to-right order (slicing, concatenation) do not work with dictionaries.

10 Creating a dictionary Python dictionaries do not support duplicate keys music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973, 'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973} print(music) {'year': 1973, 'album': 'Dark Side of the Moon', 'band': 'Pink Floyd } <class 'dict'> Python dictionaries do not support duplicate keys, so if we try to add a key having the same name with an existing one, the operation will fall in an update operation, so the already existing key will receive the value from the new key with the same name.

11 Dictionaries Accessing an element from the dictionary music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973} band = music['band'] print(band) Pink Floyd The same convention of square bracketing used for lists, but instead of indexes, we use keys Adding an element in the dictionary music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973} music['components'] = ["Roger Waters", "David Gilmour", "Richard Wright", "Nick Mason"] print(music) {'year': 1973, 'components': ['Roger Waters', 'David Gilmour', 'Richard Wright', 'Nick Mason'], 'album': 'Dark Side of the Moon', 'band': 'Pink Floyd'}

12 Dictionaries Deleting an element from dictionary music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973} del music['year'] print(music) {'album': 'Dark Side of the Moon', 'band': 'Pink Floyd'} Modifying elements in dictionary music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973} music['album'] = 'The Wall' music['year'] = 1979 print(music) {'band': 'Pink Floyd', 'year': 1979, 'album': 'The Wall'}

13 Dictionaries Obtaining all the keys in a dictionary music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973, 'band' : 'Pink Floyd', 'album' : 'The Wall', 'year' : 1979} print(music.keys()) dict_keys(['year', 'album', 'band']) Obtaining all the values in a dictionary music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973, 'band' : 'Pink Floyd', 'album' : 'The Wall', 'year' : 1979} print(values()) dict_values(['the Wall', 'Pink Floyd', 1979])

14 Dictionaries Obtaining all the keys and values pairs in a dictionary music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973, 'band' : 'Pink Floyd', 'album' : 'The Wall', 'year' : 1979} print(music.items()) dict_items([('album', 'The Wall'), ('band', 'Pink Floyd'), ('year', 1979)]) Method Python 2.7 Python 3 dict.keys() list dict_key dict.values() list dict_value dict.items() list dict_items they are called views Lists in 2.7 are just snapshots of the dictionary content. If the dictionary changes while iterating over the list, the list content remains unchanged. The views change dynamically as the dictionary changes.

15 Dictionaries Iterating over a view music = {'band' : 'Pink Floyd', 'album' : 'The Wall', 'year' : 1979} keys = music.keys() for k in keys: print(k) album year band Iterating over the items view music = {'band' : 'Pink Floyd', 'album' : 'The Wall', 'year' : 1979} keys = music.items() for k,v in keys: print(k, v) year 1979 band Pink Floyd album The Wall Note the special way in which we can use the for iterator

16 More on accessing elements from a dictionary What happens if we try to access a key that does not exist? music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973} element = music['hasdvd'] print(element) Traceback (most recent call last): File "/lect-examples.py", line 27, in <module> element = music['hasdvd'] KeyError: 'hasdvd' To avoid this situation, use get() method music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973} element_default = music.get('hasdvd') element_custom = music.get('hasdvd', 'Yes, it has') print(element_default) print(element_custom) None Yes, it has The get() method returns None or a passedin default value if the requested key does not exist

17 Operations on dictionaries Merging two dictionaries music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973} literature = {'author' : 'Erich Maria Remarque', 'book' : "The Black Obelisk"} music.update(literature) print(music) {'band': 'Pink Floyd', 'year': 1973, 'book': 'The Black Obelisk', 'author': 'Erich Maria Remarque', 'album': 'Dark Side of the Moon'} Delete an element of a dictionary using pop() music = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon', 'year' : 1973} deleted_val = music.pop('year') print(deleted_val) print(music) 1973 {'band': 'Pink Floyd', 'album': 'Dark Side of the Moon'}

18 Dictionaries Other ways to create dictionaries d1 = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon'} d2 = {} d2['band'] = 'Pink Floyd' d2['album'] = 'Dark Side of the Moon' d3 = dict(band='pink Floyd', album='dark Side of the Moon') d4 = dict([('band', 'Pink Floyd'), ('album', 'Dark Side of the Moon')]) print(d1) print(d2) print(d3) print(d4) {'album': 'Dark Side of the Moon', 'band': 'Pink Floyd'} {'album': 'Dark Side of the Moon', 'band': 'Pink Floyd'} {'album': 'Dark Side of the Moon', 'band': 'Pink Floyd'} {'album': 'Dark Side of the Moon', 'band': 'Pink Floyd'}

19 Simulate a switch statement in Python Python does not have a switch statement, but sometimes we need such functionality for the clarity of the code. Do not despair, we can simulate a switch using dictionaries. Suppose we have a program displaying a menu like below: 1 - Do something 2 - Do something else 3 - Do something different than the other options 4 - Exit

20 Switch simulation Input a number: 2 Hello from func2 def func1(): print("hello from func1") def func2(): print("hello from func2") def func3(): print("hello from func3") def func4(): print("exit") def error_handler(): print("inexistent option, retry") options = {1 : func1, 2 : func2, 3 : func3, 4 : func4} user_input = int(input("input a number: ")) func = options[user_input] func() Input a number: 7 Traceback (most recent call last): File "switch-sim.py", line 19, in <module> func = options[user_input] KeyError: 7 Option between 1-4 Option greater than 4

21 Switch simulation Input a number: 2 Hello from func2 def func1(): print("hello from func1") def func2(): print("hello from func2") def func3(): print("hello from func3") def func4(): print("exit") def error_handler(): print("inexistent option, retry") options = {1 : func1, 2 : func2, 3 : func3, 4 : func4} user_input = int(input("input a number: ")) func = options.get(user_input, error_handler) func() Input a number: 7 Inexistent option, retry Option between 1-4 Option greater than 4

22 List comprehensions List comprehension refers to a way to construct lists in a very natural, easy way, very close to the mathematical language S = [x ** 2 for x in range(0, 10)] print(s) [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] T = ['alfa', 'beta', 'gamma'] U = [x.upper() for x in T] print(u) ['ALFA', 'BETA', 'GAMMA']

23 List Comprehensions Other examples l = ['x', 'y', 'z'] comprehended = ['---' + c + '---' for c in l] print(comprehended) ['---x---', '---y---', '---z---'] Create a new list using map() function def dash(c): return '---' + c + '---' l = list(map(dash, ['x', 'y', 'z'])) print(l) ['---x---', '---y---', '---z---']

24 Dictionary Comprehensions Creating a dictionary in one step, using zip() method d = dict(zip(['album', 'band', 'year'], ['Dark Side of the Moon', 'Pink Floyd', 1973])) print(d) {'band': 'Pink Floyd', 'album': 'Dark Side of the Moon', 'year': 1973} Create a new list using map() function d = {k:v for (k,v) in zip(['album', 'band', 'year'], ['Dark Side of the Moon', 'Pink Floyd', 1973])} print(d) {'band': 'Pink Floyd', 'year': 1973, 'album': 'Dark Side of the Moon'}

25 Files Another built-in type that provides a way to interact with the computer files inside Python programs. Files are distinct from all the other built-in types, they export only some methods for common processing operations like input from and output to an external file, associated with a file object in Python programs. Open file Process data Close file Time sequence for file processing

26 Opening a file 1. The file exists and we want to read from it f = open("/myfile.txt", "r") f = open("/myfile.txt") (default is r ) 2. The file exists or not and we want to write in it f = open("/myfile.txt", "w") 3. The file exists or not and we want to append data in it 4. The file content is binary f = open("/myfile.txt", "a") f = open("/myfile.txt", "ab") if the file exists and it is opened for writing, the old content is overwritten if the file exists and it is opened for appending, data is written at the end of the file b modifier added 5. The file is opened both for reading and writing f = open("/myfile.txt", "a+") + modifier added

27 Writing into and reading from a file Writing something into a file f = open("/myfile.txt", "w") f.write("hello,\n") f.write("is there anybody in there?") f.close() Reading from the file above, line by line f = open( /myfile.txt", "r") print(f.readline()) print(f.readline()) Hello, Is there anybody in there? Reading from the file above in a string f = open( /myfile.txt", "r") print(f.read()) Hello, Is there anybody in there? Not a good idea for big files

28 Some comments regarding closing a file Do we really need to close a file after using it? Python reclaims all the memory space of the objects that are no longer referenced. So, when a file object is reclaimed, Python closes the file object automatically. Therefore, using close() method is optional. However: Closing a file manually, using close() method is a very good habit, especially in the big systems. So, as a rule of thumb, use close() every time you deal with files. Method 1 f = None f = open("/myfile.txt", "w") try: f.write("written from try/finally block") finally: if f is not None: f.close() Method 2 with open("/myfile.txt", "w") as f: f.write("this is a text.") f is an alias Using with statement, the file is closed immediately after the block execution ends.

29 Reading a text file text-file.txt "In the moment when I truly understand my enemy, understand him well enough to defeat him, then in that very moment I also love him. I think it s impossible to really understand somebody, what they want, what they believe, and not love them the way they love themselves. And then, in that very moment when I love them... I destroy them." --Andrew Ender Wiggin (Orson Scott Card - Ender's Game) file_obj = open("text-file.txt", "r") print("first pass \n + file_obj.read()) print("second_pass \n + file_obj.read()) file_obj.close() print("third pass " + file_obj.read()) What about the output of this code?

30 Program Output first pass "In the moment when I truly understand my enemy, understand him well enough to defeat him, then in that very moment I also love him. I think it s impossible to really understand somebody, what they want, what they believe, and not love them the way they love themselves. And then, in that very moment when I love them... I destroy them." --Andrew Ender Wiggin (Orson Scott Card - Ender's Game) second_pass Traceback (most recent call last): File "/Users/adriancopie/Google Drive/Python/Projects/files-and-more/example-one.py", line 17, in <module> print("third pass " + file_obj.read()) ValueError: I/O operation on closed file. Nothing printed at the second pass Why? Error at the third pass

31 Modified program file_obj = open("text-file.txt", "r") print("first pass \n" + file_obj.read()) # reset the position pointer file_obj.seek(0) # only after resetting positional pointer it is possible # to print the file content again print("second_pass \n" + file_obj.read()) file_obj.close() What about the new output of this code?

32 Program Output first pass "In the moment when I truly understand my enemy, understand him well enough to defeat him, then in that very moment I also love him. I think it s impossible to really understand somebody, what they want, what they believe, and not love them the way they love themselves. And then, in that very moment when I love them... I destroy them." --Andrew Ender Wiggin (Orson Scott Card - Ender's Game) second_pass "In the moment when I truly understand my enemy, understand him well enough to defeat him, then in that very moment I also love him. I think it s impossible to really understand somebody, what they want, what they believe, and not love them the way they love themselves. And then, in that very moment when I love them... I destroy them." --Andrew Ender Wiggin (Orson Scott Card - Ender's Game) Second pass has now the desired output

33 Other methods on file objects read(size), seek(), tell() with open("text-file.txt") as f: txt = f.read(7) print(txt) print(f.tell()) f.seek(15) print(f.tell()) txt = f.read(5) print(txt) "In the 7 15 when read(size) - tell() - reads some quantity of data and returns it as a string (in text mode) or bytes object (in binary mode) returns an integer giving the file object s current position in the file seek() - changes the file object s position

34 Reading and writing from and into files Reading a file using iterators f = open("/myfile.txt", "r") for line in f: print(line) Hello, Is there anybody in there? This is the recommended method for reading files Writing various Python objects to a file f = open("/myfile.txt", "w") day, month, year = 21, 10, 2017 lst = ['Monty', 'Python', 'and', 'the', 'Holy', 'Grail'] dct = {'band' : 'Pink Floyd', 'album' : 'Dark Side of the Moon'} f.write("today is {0}/{1}/{2}\n".format(day, month, year)) f.write(str(lst) + '\n') f.write(str(dct) + \n') f.close() File content: Today is 21/10/2017 ['Monty', 'Python', 'and', 'the', 'Holy', 'Grail'] {'album': 'Dark Side of the Moon', 'band': 'Pink Floyd'}

35 Copying two files Copying a text file with open('text-file.txt', 'r') as rf: with open('copy-text-file.txt', 'w') as wf: for line in rf: wf.write(line) Copying a binary file (version 1) with open('binary-file.png', 'rb') as rf: with open('copy-binary-file.png', 'wb') as wf: for line in rf: wf.write(line) Copying a binary file (version 2) with open('binary-file.png', 'rb') as rf: with open('copy-binary-file.png', 'wb') as wf: chunk_size = 4096 read_chunk = rf.read(chunk_size) while len(read_chunk) > 0: wf.write(read_chunk) read_chunk = rf.read(chunk_size)

36 More on list comprehension on files Suppose we need to create a list from the content of a file. The file s content looks as below: Version 1 Note that after reading a line from a file, a new line char is appended at the end of the line File content: Monty Python and the Holy Grail l = [] with open('myfile.txt') as f: for x in f: l.append(x) print(l) ['Monty\n', 'Python\n', 'and\n', 'the\n', 'Holy\n', 'Grail']

37 More on lists comprehensions on files Version 2 with list comprehension q = [] with open('myfile.txt') as f: q = [x.strip() for x in f] print(q) ['Monty', 'Python', 'and', 'the', 'Holy', 'Grail'] Version 3 with list comprehension r = [x.strip() for x in open( myfile.txt')] print(r) ['Monty', 'Python', 'and', 'the', 'Holy', 'Grail']

38 List comprehension on files, with filtering Suppose we need to create a list from the content of a file, but only with some of the lines, fulfilling a certain condition. Example: based on the content of the file in the previous example, build a list containing only the lines beginning with M and P r = [x.strip() for x in open('myfile.txt') if x[0] == 'M' or x[0] == 'P'] print(r) ['Monty', 'Python'

39 Comparison Python always inspects all the compound parts of the objects until it can come with a result in what concerns the equality of two objects. If the objects are compound, Python traverses all the structures, applying left to right comparisons until a first difference is encountered. list1 = [1, 2, 3] list2 = [1, 2, 3] print(list1 == list2) print(list1 is list2) True False The == operator tests value equivalence The is operator tests object identity list1 = [1, 2, 3] list2 = [1, 2, 3] print(id(list1)) print(id(list2))

40 Comparison Relative magnitude comparison: list1 = [1, ('x', 1)] list2 = [1, ('x', 2)] print(list1 < list2) print(list1 == list2) print(list1 > list2) True False False Numbers - compared by relative magnitude Strings - compared lexicographically Lists and tuples - comparing each element, from left to right and recursively for nested structures, until a mismatch is found Sets - they are equal if they contain the same items Dictionaries - they are equal if their sorted (key, value) lists are equal

41 True and False in Python Numbers are false if zero, true otherwise Other objects are false if empty, true otherwise Object Value python" True " False [1, 2] True [] False { a : 1} True {} False 1 True 0 False None False

42 The None object None is a special object in Python and it always considered to be false It serves as an empty placeholder (similar with NULL in C or null in Java) Use case: suppose you want to assign an element in a list at a specified offset. But you can perform this operation only if that offset exists, otherwise an index out of bound error occurs. Solution: preallocate the list with None objects. l = [None] * 5 l[2] = 2 print(l) [None, None, 2, None, None] Note: The list could be preallocated with any other value (like 0-es) but the good practice recommends the use of None. None does not mean undefined. It is a real object, occupying a real memory address. print(sys.getsizeof(l[0])) print(sys.getsizeof(l[1])) print(sys.getsizeof(l[2]))

43 Lists repetition Repeating a list is similar with adding it to itself multiple times. However, when it comes to deal with nested mutable sequences, the effect is different. l = [1, 2, 3] q = l * 4 r = [l] * 4 print(q) print(r) l[1] = 0 print(q) print(r) [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3] [[1, 2, 3], [1, 2, 3], [1, 2, 3], [1, 2, 3]] [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3] [[1, 0, 3], [1, 0, 3], [1, 0, 3], [1, 0, 3]]

44 Changing a list Suppose we want to change a list by incrementing its every element by 1 l = [1, 2, 3, 4] for i in l: i += 1 print(l) [1, 2, 3, 4]? What happened?

45 Changing a list We can solve this issue by using indexes l = [1, 2, 3, 4] for i in range(len(l)): l[i] += 1 print(l) [2, 3, 4, 5] range() generates indexes, so we can use positional list update i = 0 l[0] = l[0] = 2

46 Conclusions Conclusions Dictionaries are the most flexible types in Python Lists and dictionary comprehensions offer a method for effectively creating lists and dictionaries Files are another type in Python, exposing methods for reading and writing Object are compared for equality based on specific rules