Lab 5: Function Types, Lists and Dictionaries, Sorting. Ling 1330/2330: Computational Linguistics Na-Rae Han

Transcription

1 Lab 5: Function Types, Lists and Dictionaries, Sorting Ling 1330/2330: Computational Linguistics Na-Rae Han

2 Objectives Assignment review Pig Latin Tokenization function Two types of function: returning vs. void Building a list, list methods Dictionary, dict methods Counting with dict Sorting a sequence with sorted() 9/11/2018 2

3 Pig Latin def vowelindex(wd) : def getinitialcs(wd) : return wd[:vowelindex(wd)] def gettherest(wd) : return wd[vowelindex(wd):] Showcases strength of functions. A fairly complex behavior has been modeled in a neatly modular fashion. def piglatinword(wd) : if wd[0] in 'aeiou' : return wd + 'way' else : return gettherest(wd) + getinitialcs(wd) + 'ay' def piglatinsent(sent) : piglatinword('') Traceback (most recent call last): if wd[0] in 'aeiou': IndexError: string index out of range The function has a critical flaw. What? 9/11/2018 3

4 Pig Latin: more robust def vowelindex(wd) : def getinitialcs(wd) : return wd[:vowelindex(wd)] def gettherest(wd) : return wd[vowelindex(wd):] def piglatinword(wd) : if len(wd) == 0 : return '' elif wd[0] in 'aeiou' : return wd + 'way' else : return gettherest(wd) + getinitialcs(wd) + 'ay' def piglatinsent(sent) : Addresses fringe cases. Function is more robust. piglatinword('') '' Any other improvements? 9/11/2018 4

5 Pig Latin: 'y' is a vowel sometimes piglatinword('yellow') 'ellowyay' piglatinword('ploy') 'oyplay' Good. 'y' is a consonant Also good. 'y' is a part of a diphthong piglatinword('python') 'onpythay' vowelindex('python') 4 getinitialcs('python') 'pyth' Simply adding 'y' in 'aeiou' will not work. (WHY?) Not good. 'y' is a vowel here but not treated as one. To properly handle words with 'y' as the initial vowel, vowelindex() function must be further refined. 9/11/2018 5

6 Better tokenization sent = 'rose is a rose is a rose is a rose.' sent.split() ['rose', 'is', 'a', 'rose', 'is', 'a', 'rose', '.'] sent = 'Rose is a rose is a rose is a rose.' sent.split() ['Rose', 'is', 'a', 'rose', 'is', 'a', 'rose.'] joke = "Knock knock, who's there?" joke.split() ['Knock', 'knock,', "who's", 'there?'].split() is too simple as a tokenizer. We need a better tokenizer: - folds case - separates out symbols 9/11/2018 6

7 Lower, space out symbols, then split "Knock knock, who's there?".lower().replace().split() "knock knock, who's there?" "knock knock, who ' s there? " ['knock', 'knock', ',', 'who', "'", 's', 'there', '?'] 9/11/2018 7

8 Tokenizer function, first try A function that tokenizes a sentence: def gettokens(sent) : new = sent.lower() new = new.replace(",", ", ").replace("'", " ' ").replace("?", "? ") return new.split().replace("'", " ' ") inserts a space before and after ' joke = "Knock knock, who's there?" gettokens(joke) ['knock', 'knock', ',', 'who', "'", 's', 'there', '?'] Why " ' " instead of just " '"? Success! '?' is its own word. 9/11/2018 8

9 Tokenizer function, first try A function that tokenizes a sentence: def gettokens(sent) : new = sent.lower() new = new.replace(",", ", ").replace("'", " ' ").replace("?", "? ") return new.split() But there are so many more symbols:. : # $! joke = "Knock knock, who's there?" gettokens(joke) ['knock', 'knock', ',', 'who', "'", 's', 'there', '?'] This replace line is about to become a mile long! 9/11/2018 9

10 Tokenizer function, second try A function that tokenizes a sentence: def gettokens(sent) : new = sent.lower() for s in ",.!?':;#$" : new = new.replace(s, " "+s+" ") return new.split().replace(s, " "+s+" ") inserts a space before and after each symbol s joke = "Knock knock, who's there?" gettokens(joke) ['knock', 'knock', ',', 'who', "'", 's', 'there', '?'] pal = "A man, a plan, a canal: Panama." gettokens(pal) ['a', 'man', ',', 'a', 'plan', ',', 'a', 'canal', ':', 'panama', '.'] Looks good! 9/11/

11 Tokenizer function 2 minutes A function that tokenizes a sentence: def gettokens(sent) : new = sent.lower() for s in ",.!?':;#$" : new = new.replace(s, " "+s+" ") return new.split() joke = "Knock knock, who's there?" gettokens(joke) ['knock', 'knock', ',', 'who', "'", 's', 'there', '?'] pal = "A man, a plan, a canal: Panama." gettokens(pal) ['a', 'man', ',', 'a', 'plan', ',', 'a', 'canal', ':', 'panama', '.'] 9/11/

12 Value-returning functions vs. Void functions (aka "printing" functions) 9/11/

13 Two function types: returning vs. void def ahoyfy(word) : return word.replace('o', '-ahoy-') def ahoyprint(word) : print(word.replace('o', '-ahoy-')) return x print(x) ahoyfy('cola') 'c-ahoy-la' ahoyprint('cola') c-ahoy-la Seems to work similarly. What's the difference? 9/11/

14 Returning vs. printing def ahoyfy(word) : return word.replace('o', '-ahoy-') def ahoyprint(word) : print(word.replace('o', '-ahoy-')) return x print(x) ahoyfy('cola') 'c-ahoy-la' ahoyprint('cola') c-ahoy-la RETURNS a string (Note the quotes '') PRINTS a string 9/11/

15 Only returned value can be assigned def ahoyfy(word) : return word.replace('o', '-ahoy-') def ahoyprint(word) : print(word.replace('o', '-ahoy-')) return x print(x) foo = ahoyfy('cola') foo 'c-ahoy-la' foo = ahoyprint('cola') c-ahoy-la foo foo is assigned to the returned string Printing happens during assignment statement, and 9/11/

16 Void functions implicitly return None def ahoyfy(word) : return word.replace('o', '-ahoy-') def ahoyprint(word) : print(word.replace('o', '-ahoy-')) return None When there is no return statement, function implicitly returns None. foo = ahoyfy('cola') foo 'c-ahoy-la' foo = ahoyprint('cola') c-ahoy-la foo foo's value is nothing! 9/11/

17 Returned values can be reused def ahoyfy(word) : return word.replace('o', '-ahoy-') def ahoyprint(word) : print(word.replace('o', '-ahoy-')) ahoyfy('cola').upper() 'C-AHOY-LA' len(ahoyfy('cola')) 9 len(ahoyprint('cola')) ahoyfy returns string, can be fed into another function that takes string input c-ahoy-la Traceback (most recent call last): File "<pyshell#17>", line 1, in <module> len(ahoyprint('cola')) TypeError: object of type 'NoneType' has no len() ahoyprint returns None, which has no length! 9/11/

18 Python data types int: integer float: floating point number 'Bart' 'Hello, world!' str: string (a piece of text) ['cat', 'dog', 'fox', 'hippo'] list ('gold', 'silver', 'bronze') tuple {'Homer':36, 'Marge':36, 'Bart':10, 'Lisa':8, 'Maggie':1} dict: (dictionary) maps a value to an object 9/11/

19 Changing list entry li = [1,2,3] li [1, 2, 3] li[2] = 10 li [1, 2, 10]?? ani = ['fox', 'owl', 'dog'] ani ['fox', 'owl', 'dog']?? ani[1] = 'bat' ani ['fox', 'bat', 'dog'] 9/11/

20 Changing list entry li = [1,2,3] li [1, 2, 3] li[2] = 10 li [1, 2, 10] ani = ['fox', 'owl', 'dog'] ani ['fox', 'owl', 'dog'] ani[1] = 'bat' ani ['fox', 'bat', 'dog'] A new assignment statement updates an existing list entry 9/11/

21 Adding to a list li = [1,2,3] li [1, 2, 3]?? li.append(4) li [1, 2, 3, 4] ani = ['fox', 'owl', 'dog'] ani ['fox', 'owl', 'dog']?? ani.append('cat') ani ['fox', 'owl', 'dog', 'cat'] 9/11/

22 Adding to a list li = [1,2,3] li [1, 2, 3] li.append(4) li [1, 2, 3, 4] ani = ['fox', 'owl', 'dog'] ani ['fox', 'owl', 'dog'] ani.append('cat') ani ['fox', 'owl', 'dog', 'cat'].append(x) adds a single item at the end 9/11/

23 .append() syntax li = [1,2,3] li [1, 2, 3] li.append(4) li [1, 2, 3, 4] ani = ['fox', 'owl', 'dog'] ani ['fox', 'owl', 'dog'] ani.append('cat') ani ['fox', 'owl', 'dog', 'cat'] Note the syntax!! li.append(4) This part is never, ever necessary for.append() li = li.append(4) 9/11/

24 Pig Latin sentence translator def piglatinsent(sent) : """Takes a sentence, returns its Pig Latin translation. """ tran = [] for w in sent.lower().replace('.', '').split(): tran.append(piglatinword(w)) return ' '.join(tran).capitalize() + '.' 9/11/

25 Adding to a list li = [1,2,3] li [1, 2, 3] li.append(4) li [1, 2, 3, 4]???? li.extend([5,6,7]) li [1, 2, 3, 4, 5, 6, 7]?? li.insert(1, 1.5) li [1, 1.5, 2, 3, 4, 5, 6, 7] 9/11/

26 Adding to a list li = [1,2,3] li [1, 2, 3] li.append(4) li [1, 2, 3, 4] li.extend([5,6,7]) li [1, 2, 3, 4, 5, 6, 7] li.insert(1, 1.5) li [1, 1.5, 2, 3, 4, 5, 6, 7].append(x) adds a single item at the end.extend(list) adds a list of items at the end.insert(i,x) inserts an item at index i 9/11/

27 .append() vs..extend() li = [1, 2] li.append(3) li [1, 2, 3] li.extend([4,5]) li [1, 2, 3, 4, 5] li.append([6,7]) li [1, 2, 3, 4, 5, [6, 7]] len(li) 6 List inside a list! [6,7] is appended as a single element. li has length 6 9/11/

28 .extend() vs. + li = [1, 2, 3] li.extend([4, 5, 6]) li [1, 2, 3, 4, 5, 6] li + [7, 8, 9] [1, 2, 3, 4, 5, 6, 7, 8, 9] li [1, 2, 3, 4, 5, 6] + also extends a list, but it creates and returns a NEW list. li is NOT affected. 9/11/

29 More list methods.index().count().remove().pop() No time to cover in class. You should learn them on your own. Study the Python Tutorials, etc. 9/11/

30 dict: a dictionary data type {'Homer':36, 'Marge':36, 'Bart':10, 'Lisa':8, 'Maggie':1} A dictionary of the Simpson family members' age {'go':'went', 'eat':'ate', 'see':'saw', 'say':'said'} A dictionary of verb past tense Dictionaries store a mapping between a set of keys and a set of values. Keys can be any immutable type: string, integer, tuple Values can be any type (can also be mixed types) There is no inherent order (unlike lists and tuples) You can define, modify, view, lookup, and delete the key-value pairs in the dictionary. 9/11/

31 No order, dictionary lookup en2es = {'cat':'gato', 'dog':'perro', 'tiger':'tigre'} en2es {'cat': 'gato', 'dog': 'perro', 'tiger': 'tigre'} en2es['cat'] 'gato' en2es['dog'] 'perro' Looking up is done through key Don't be fooled! There is no real order. 9/11/

32 Dictionary lookup errors en2es = {'cat':'gato', 'dog':'perro', 'tiger':'tigre'} en2es {'cat': 'gato', 'dog': 'perro', 'tiger': 'tigre'} en2es['platypus'] Traceback (most recent call last): File "<pyshell#31>", line 1, in <module> en2es['platypus'] KeyError: 'platypus' en2es['gato'] Traceback (most recent call last): File "<pyshell#32>", line 1, in <module> en2es['gato'] KeyError: 'gato' Key does not exist in dictionary Dictionary is one way. Cannot look up based on the value. Mapping can be many-to-one 9/11/

33 Checking if something's in a dictionary en2es {'cat': 'gato', 'dog': 'perro', 'tiger': 'tigre'} 'platypus' in en2es False 'cat' in en2es True "in" tests if a key is in a dictionary 'gato' in en2es False "in" does not work with value 9/11/

34 Practice: getplural() 2 minutes Write a "pluralizer" function plu = {'man':'men', 'goose':'geese', 'mouse':'mice', 'child':'children'} def getplural(noun) : if noun in plu : return plu[noun] else: return noun + 's'?? getplural('cat') 'cats' getplural('goose') 'geese' 9/11/

35 Practice: getplural() Write a "pluralizer" function plu = {'man':'men', 'goose':'geese', 'mouse':'mice', 'child':'children'} def getplural(noun) : if noun in plu : return plu[noun] else : return noun + 's' If noun is in the irregular pl noun dictionary, return the dict value getplural('cat') 'cats' getplural('goose') 'geese' 9/11/

36 Updating vs. adding new entry 35 {'Homer': 36, 'Marge': 36, 'Bart': 10, 'Lisa': 8} {'Homer': 36, 'Marge': 36, 'Bart': 10, 'Lisa': 8, 'Maggie': 1} sim = {'Homer':36, 'Marge':35, 'Bart':10, 'Lisa':8} sim['marge'] sim['marge'] = 36 sim sim['maggie'] = 1 sim key does not exist! Creates a new key and gives value key already exists. Updates its value. 9/11/

37 Deleting entry {'Homer': 36, 'Marge': 36, 'Bart': 10, 'Lisa': 8, 'Maggie': 1} {'Homer': 36, 'Marge': 36, 'Lisa': 8, 'Maggie': 1} sim del sim['bart'] sim del deletes a key and its value from a dictionary 9/11/

38 Finding out what's in sim = {'Lisa':8, 'Marge':35, 'Bart':10, 'Homer':36} sim.keys()?? dict_keys(['lisa', 'Marge', 'Bart', 'Homer']) sim.values()?? dict_values([8, 35, 10, 36]) 9/11/

39 Finding out what's in sim = {'Lisa':8, 'Marge':35, 'Bart':10, 'Homer':36} sim.keys() dict_keys(['lisa', 'Marge', 'Bart', 'Homer']) sim.values() dict_values([8, 35, 10, 36]).keys() returns a quasi-list of keys,.values() returns a quasi-list of values. The orders match! 9/11/

40 .keys().values(): list-like, but not quite sim.keys() dict_keys(['lisa', 'Marge', 'Bart', 'Homer']) dict_values([8, 35, 10, 36]) sim = {'Lisa':8, 'Marge':35, 'Bart':10, 'Homer':36} sim.values() for v in sim.values() : print(v) len(sim.keys()) sum(sim.values()) Quasi-lists: These behave very much like a list, but not always sim.keys()[2] Traceback (most recent call last): File "<pyshell#56>", line 1, in <module> sim.values()[2] TypeError: 'dict_keys' object does not support indexing 9/11/

41 .items() to reference (key, value) pairs sim {'Lisa': 8, 'Marge': 35, 'Bart': 10, 'Homer': 36} sim.items() dict_items([('lisa', 8), ('Marge', 35), ('Bart', 10), ('Homer', 36)]).items() returns a quasi-list of (key, value) pairs 9/11/

42 Iterating through (key, value) pairs sim {'Lisa': 8, 'Marge': 35, 'Bart': 10, 'Homer': 36} sim.items() dict_items([('lisa', 8), ('Marge', 35), ('Bart', 10), ('Homer', 36)]) for (k, v) in sim.items() : print(k, 'is', v, 'years old.') Lisa is 8 years old. Marge is 35 years old. Bart is 10 years old. Homer is 36 years old..items() lets you reference key and its value side by side as (k, v) in a for loop! 9/11/

43 Iterating through keys sim {'Lisa': 8, 'Marge': 35, 'Bart': 10, 'Homer': 36} for k in sim : print(k, 'is', sim[k], 'years old.') Lisa is 8 years old. Marge is 35 years old. Bart is 10 years old. Homer is 36 years old. Looping through keys then accessing value through dict[key]. Does the same thing! 9/11/

44 Common dict application: counting 3 tally = {'gold':1, 'bronze':3} tally['bronze'] newmedals = ['bronze', 'silver', 'gold', 'gold', 'silver'] for m in newmedals : tally[m] += 1 Traceback (most recent call last): File "<pyshell#80>", line 2, in <module> tally[m] += 1 KeyError: 'silver' Error: 'silver' is not yet in the dictionary as a key 9/11/

45 Initial key created, value set to 1 3 tally = {'gold':1, 'bronze':3} tally['bronze'] newmedals = ['bronze', 'silver', 'gold', 'gold', 'silver'] for m in newmedals : if m not in tally : else : tally[m] = 1 tally[m] += 1 Make sure to account for the initial key creation & assignment tally {'bronze': 4, 'gold': 3, 'silver': 2} 9/11/

46 Dictionary of frequency count chars = list('abracadabra') chars ['a', 'b', 'r', 'a', 'c', 'a', 'd', 'a', 'b', 'r', 'a'] freq = {} for c in chars: if c in freq: freq[c] += 1 else: freq[c] = 1 freq {'r': 2, 'a': 5, 'b': 2, 'c': 1, 'd': 1} Initialize an empty dictionary For loop through list, tally up the counts 9/11/

47 Try it out 2 minutes chars = list('abracadabra') chars ['a', 'b', 'r', 'a', 'c', 'a', 'd', 'a', 'b', 'r', 'a'] freq = {} for c in chars: if c in freq: freq[c] += 1 else: freq[c] = 1 freq {'r': 2, 'a': 5, 'b': 2, 'c': 1, 'd': 1} for (k, v) in freq.items(): print(k, v) for k in freq: print(k, freq[k]) also try 9/11/

48 Sorting a list with sorted() sorted(list) li = [2, 3, 1, 4] sorted(li) [1, 2, 3, 4] li [2, 3, 1, 4] ani = ['fox', 'owl', 'dog', 'cat'] sorted(ani) ['cat', 'dog', 'fox', 'owl'] ani ['fox', 'owl', 'dog', 'cat'] 9/11/

49 Sorting a list with sorted() sorted(list) li = [2, 3, 1, 4] sorted(li) [1, 2, 3, 4] li [2, 3, 1, 4] Creates and returns a new sorted list li is not changed 9/11/

50 Sorting a list with sorted() sorted(list) Creates and returns a new alphabetically sorted list ani = ['fox', 'owl', 'dog', 'cat'] sorted(ani) ['cat', 'dog', 'fox', 'owl'] ani ['fox', 'owl', 'dog', 'cat'] ani is unchanged 9/11/

51 Sorting anything with sorted() sorted() is a general function: it works on any sequence types (strings & tuples) and dictionaries. No matter the input type, it returns a list. seasons = ('spring', 'summer', 'fall', 'winter') sorted(seasons, reverse=true) ['winter', 'summer', 'spring', 'fall'] takes a tuple, returns a sorted list sorted('python') ['h', 'n', 'o', 'p', 't', 'y'] takes a string, returns a sorted list of characters sim = {'Homer':36, 'Marge':35, 'Bart':10, 'Lisa':8} sorted(sim) ['Bart', 'Homer', 'Lisa', 'Marge'] takes a dictionary, returns a sorted list of *keys* 9/11/

52 Teaching yourself new tricks Python built-in helper functions dir() help() Python IDLE tool tips Online references Python 3 Quick Reference: ython3-v1.0.5a-english.pdf 9/11/

53 dir() and help() dir(str) [' add ', ' class ', ' contains ', ' delattr ',... ' subclasshook ', '_formatter_field_name_split', 'capitalize', 'center', 'count', 'decode', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'index', 'isalnum', 'isalpha', 'isdigit', 'islower', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill'] dir(obj) Returns a list of attributes ( xyz ) and methods that are available for the given object. 9/11/

54 dir() and help() dir(str) [' add ', ' class ', ' contains ', ' delattr ',... ' subclasshook ', '_formatter_field_name_split', 'capitalize', 'center', 'count', 'decode', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'index', 'isalnum', 'isalpha', 'isdigit', 'islower', 'isspace', 'istitle', help(str.find) 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'partition', Help method_descriptor: 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', find(...) 'strip', 'swapcase', 'title', 'translate', 'upper', S.find(sub 'zfill'] [,start [,end]]) -> int Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation. Return -1 on failure. help(obj.method) prints out information on the object's method 9/11/

55 Wrap-up Next class: Language and Computers, Ch.2 Writer's aids: spell checkers Exercise 4 Due Thu, 4pm 9/11/