Data Structures - CSCI 102. CS102 Hash Tables. Prof. Tejada. Copyright Sheila Tejada

Transcription

1 CS102 Hash Tables Prof. Tejada 1

2 Vectors, Linked Lists, Stack, Queues, Deques Can t provide fast insertion/removal and fast lookup at the same time The Limitations of Data Structure Binary Search Trees, Heaps Provide consistently fast operations, but must maintain an internal ordering How can we further improve the performance of lookup, add & removal? What if we didn t care about the ordering of the elements at all? 4

3 Lookup Tables For operations where we only care about fast add/remove/search, not fast traversal, we create a table structure to optimize for fast lookup Each value in the table has a unique key The key is used as a short identifier to lookup an entire value in the table Example Your student ID is used to look up your student record (e.g. name, GPA, etc.) 5

4 Lookup Tables What kind of operations do we need to perform on a lookup table? Search(key) See if a particular value identified by key is in the table Insert(key,value) Insert a new value identified by key into the table Remove(key) Remove the value identified by key from the table We don t care as much about traversal (visiting all elements) in this scenario 6

5 Sample Object We want to keep a directory of all the students at USC and be able to look them up by their student ID Let s assume ID is a unique integer struct Student { string name; double gpa; int id; }; 7

6 Direct Address Table If we can guarantee that student IDs will always range from 0 to N (e.g. 0 to 4999), we could just store them in an array: Student data[4999]; Then when we want to grab a particular student, we know Student N is at index N: int id = 3285; Student s = data[id]; 8

7 Direct Address Table Student IDs Data Student Objects John Doe Jane Doe ID 3.7 GPA Some Guy Name

8 Direct Address Table Direct Addressing Maps keys directly to the indexes in an array Unused array indexes need to be marked Generally use NULL Operations are fast O(1) worst case 10

9 Direct Address Table Direct Addressing Issues Key Restrictions Keys must fall into a nice, uniform range Keys must be numeric Array Size If there are N possible keys, then data[] must be of size N Our array could get HUGE What if we re only using a small numbers of keys? Tons of space is wasted How can we get around these limitations? 11

10 Hash Functions Hash Functions A function that maps key values to array indexes Input records all have a unique key The hash function maps key to an array index Records are stored at data[hash(key)] Ideally every unique key also has unique hash(key) Direct Addressing essentially uses a hash function that does nothing int directaddresshash(int studentid) { return studentid; } 12

11 Hash Tables Data Student Objects Student IDs (Keys) hash(4) hash(0) John Doe Jane Doe Hash Function hash(2) 4 ID 3.7 GPA Some Guy Name 13

12 Hash Tables Hash Functions How can we avoid having to make our array gigantic to hold all possible keys? 14

13 Hash Tables Hash Functions How can we avoid having to make our array gigantic to hold all possible keys? Simple solution: use modular arithmetic Size of the backing array is no longer dependent on the number of unique keys int modularhash(int studentid) { return studentid % ARRAY_SIZE; } Recall direct addressing: int directaddresshash(int studentid) { return studentid; } 15

14 Hash Functions What makes a good hash function? Fast Hashing is supposed to be faster than a binary search tree. hash(key) needs to be O(1) Deterministic If we have a key K, then hash(k) must always give the same result Uniform distribution The hash function should uniformly distribute keys across all of the available indexes in the storage array Making a good hash function is hard 16

15 Hash Functions Making a hash function Map your data into the set of natural numbers N = {0, 1, 2,...} For strings, use things like ASCII letter codes Prime numbers are your friend Prime table sizes tend to yield better results Handle variants of the same pattern E.g. make sure "get" and "gets" hash differently Try to be independent of any patterns that may exist in the data You won t usually have to write your own, but you should know what the default hash function does 17

16 Sample Hash Function The Java String hash Changing the multiplier to 32 from 31 results in significantly worse performance Primes numbers good, powers of 2 bad hash(s) = s[0] 31 n-1 + s[1] 31 n s[n-2] 31 + s[n-1] The Java Default hash function Return the address in memory of the object Works better in Java because everything is actually a pointer/reference behind the scenes int hash(string s) { int h = 0; for (i=0; i < s.size(); i++) { h = (h*31)+s[i]; } return h; } 18

17 Hash Tables Hashing Issues Hash Tables do not maintain any ordering of their internal elements Creating a perfect hash function is almost impossible Collisions When two distinct keys generate the same hash value it s called a collision hash(k1) == hash(k2) 19

18 Collision Handling Possible Collision Handling Solutions Ideal hash function Table Resizing Open Addressing Linear probing Quadratic probing Double hashing Chaining Universal Hashing Perfect Hashing 20

19 Collision Handling Ideal Hash Function Create a hash function so good that it will never cause a collision You can get close if you use a cryptographic hash algorithm like sha1 or md5 You will pay a performance penalty that will affect your insertion speed You will not get a collision if the size of your hash table is really really really large Most likely, this cannot be achieved 21

20 Collision Handling Table Resizing If your hash function uses modular arithmetic, make the array bigger when you get a collision When a collision happens... Dynamically create a new, larger array (usually double the size) Re-hash all input values into the new larger array With a little luck, no more collisions A helpful strategy, but not a very good one on its own 22

21 Collision Handling Open Addressing If we try to insert a new element and there s a collision, keep probing the hash table until we find a vacant space Probing If a collision occurs, use a deterministic algorithm to calculate the next array index to check (based on the initial hash result) All data is stored directly in the hash table. No extra data structures are needed. 23

22 Open Addressing Linear Probing If there is no collision, insert the element into the table as normal If a collision occurs, start walking through the hash table index by index looking for an empty slot First check data[hash(key)] If it s taken, check data[hash(key)+1] If it s taken, check data[hash(key)+2] Etc. If you walk off the end of the array, wrap around back to the beginning of the array 24

23 Open Addressing (Linear Probing) Start with an empty Hash Table Data

24 Open Addressing (Linear Probing) Insert "John Doe" with ID = 123 Data Student 4 Name GPA ID John Doe

25 Open Addressing (Linear Probing) Insert "John Doe" with ID = 123 hash(123) = 1 Data 0 Student hash() hash(123) = Name GPA ID John Doe

26 Open Addressing (Linear Probing) Insert "John Doe" with ID = 123 hash(123) = 1 data[1] is empty, no collision Data 0 Student hash() hash(123) = Name GPA ID John Doe

27 Open Addressing (Linear Probing) Insert "John Doe" with ID = 123 hash(123) = 1 data[1] is empty, no collision store it there Student hash() hash(123) = 1 Data John Doe Name GPA ID John Doe

28 Open Addressing (Linear Probing) Hash Table contains one item Data 0 1 John Doe

29 Open Addressing (Linear Probing) Insert "Jane Doe" with ID = 202 Name GPA ID Student Jane Doe Data John Doe

30 Open Addressing (Linear Probing) Insert "Jane Doe" with ID = 202 hash(202) = 3 Data 0 1 John Doe Name GPA ID Student Jane Doe hash() hash(202) =

31 Open Addressing (Linear Probing) Insert "Jane Doe" with ID = 202 hash(202) = 3 data[3] is empty, no collision Data 0 1 John Doe Name GPA ID Student Jane Doe hash() hash(202) =

32 Open Addressing (Linear Probing) Insert "Jane Doe" with ID = 202 hash(202) = 3 data[3] is empty, no collision store it there Data 0 1 John Doe Student hash() hash(202) = Jane Doe Name Jane Doe GPA 3.4 ID

33 Open Addressing (Linear Probing) Hash Table contains two items Data John Doe Jane Doe

34 Open Addressing (Linear Probing) Insert "Some Guy" with ID = 401 Name GPA ID Student Some Guy Data John Doe Jane Doe

35 Open Addressing (Linear Probing) Insert "Some Guy" with ID = 401 hash(401) = 1 Student hash() hash(401) = 1 Data John Doe Jane Doe Name Some Guy GPA 3.5 ID

36 Open Addressing (Linear Probing) Insert "Some Guy" with ID = 401 hash(401) = 1 data[1] is non-empty, collision! Student hash() hash(401) = 1 Data John Doe Jane Doe Name Some Guy GPA 3.5 ID

37 Open Addressing (Linear Probing) Insert "Some Guy" with ID = 401 hash(401) = 1 data[1] is non-empty, collision! Name hash(401)+1 = 2 Student Some Guy hash() hash(401) = 1 Data John Doe Jane Doe GPA ID

38 Open Addressing (Linear Probing) Insert "Some Guy" with ID = 401 hash(401) = 1 data[1] is non-empty, collision! Name hash(401)+1 = 2 data[2] is empty, no collision Student Some Guy hash() hash(401) = 1 Data John Doe Jane Doe GPA ID

39 Open Addressing (Linear Probing) Insert "Some Guy" with ID = 401 hash(401) = 1 Name data[1] is non-empty, collision! hash(401)+1 = 2 data[2] is empty, no collision store it there Student Some Guy GPA 3.5 hash() hash(401) = 1 Data John Doe Some Guy Jane Doe ID

40 Open Addressing (Linear Probing) Hash Table contains three items Data John Doe Some Guy Jane Doe

41 Open Addressing (Linear Probing) How do the following operations work now? Search(key) 1) Look for the value at data[hash(key)] If data[hash(key)] is empty, search fail If data[hash(key)] has value, search success If data[hash(key)] has different value, start linear probing 2) Repeatedly look at data[hash(key)+i] If data[hash(key)+i] is empty, search fail If data[hash(key)+i] has value, search success If data[hash(key)+i] has different value, increment i and repeat #2 If you eventually return back to the original hash(key) index, search fail 43

42 Open Addressing (Linear Probing) How do the following operations work now? Insert(key,value) Almost identical to search Start at hash(key) in the array and walk through until you find an empty slot Insert the item at the empty slot What if there are no empty slots in the table (the table is full)? Make a new, bigger table Rehash all the existing entries Go back to business as usual 44

43 Open Addressing (Linear Probing) How do the following operations work now? Remove(key) Search for the desired value Remove the value once you find it Complications You can t just set the removed value to NULL You might break other linear probe searches Have to mark elements as deleted or something similar instead 45

44 Open Addressing (Linear Probing) What is the Big O of each of these operations? Search(key) Insert(key,value) Remove(key) 46

45 Open Addressing (Linear Probing) The world s worst hash function int hash(t key) { return 0; } What would this hash function cause to happen if we were using an open addressing approach with linear probing? 47

46 Open Addressing (Linear Probing) What is the Big O of each of these operations? Search(key) Average: O(1), Worst Case: O(N) Insert(key,value) Average: O(1), Worst Case: O(N) Remove(key) Average: O(1), Worst Case: O(N) Operations depend on the table s load factor How big is the table? How many slots are taken already? load factor = (# of elements) / (size of array) "Utilization" 48

47 Open Addressing (Linear Probing) Other issues with Linear Probing Promotes clustering Long runs of occupied slots build up and increase average search time Potential Solutions Quadratic Probing Use a polynomial to determine where to look for empty slots Double Hashing Use a secondary hash function to determine where to look for empty slots 49

48 Collision Handling Chaining Each slot in the Hash Table can now contain a list of elements instead of a single element When multiple items hash to the same slot, they are placed in the list at that slot This requires the overhead of an extra list for each slot that contains one or more elements 50

49 Hash Table contains two items Chaining Data John Doe Jane Doe

50 Insert "Some Guy" with ID = 401 Chaining Data 0 John Doe Name GPA Student Some Guy Jane Doe ID

51 Chaining Insert "Some Guy" with ID = 401 hash(401) = 1 Data 0 John Doe hash(401) = 1 1 Name GPA Student Some Guy 3.5 hash() Jane Doe ID

52 Chaining Insert "Some Guy" with ID = 401 hash(401) = 1 data[1] is non-empty, collision! Data 0 John Doe hash(401) = 1 1 Name GPA Student Some Guy 3.5 hash() Jane Doe ID

53 Insert "Some Guy" with ID = 401 hash(401) = 1 data[1] is non-empty, collision! Chaining says to add the new entry to the list at data[1] Name GPA Student Some Guy 3.5 hash() Chaining hash(401) = 1 Data John Doe Jane Doe ID

54 Insert "Some Guy" with ID = 401 hash(401) = 1 data[1] is non-empty, collision! Chaining says to add the new entry to the list at data[1] Name GPA Student Some Guy 3.5 hash() Chaining Insert Some Guy in the list at data[1] hash(401) = 1 Data John Doe Jane Doe ID

55 Hash Table contains three items Chaining Some Guy Data John Doe Jane Doe

56 Chaining How do the following operations work now? Search(key) 1) Retrieve the list at data[hash(key)] If data[hash(key)] is empty, search fail 2) Linear search for the element with key in the list that was retrieved If the list contains the value, search success If the list does not contain the value, search fail 58

57 Chaining How do the following operations work now? Insert(key,value) Almost identical to search Retrieve the list at data[hash(key)] in the array If there is no list, make a one item long list containing value and re-insert it If there is a list already, add value to the list Does the table ever need to be resized? Theoretically, no...chains can grow forever In reality, sometimes it s good to resize so the chains don t become prohibitively long 59

58 Chaining How do the following operations work now? Remove(key) Almost identical to search Retrieve the list at data[hash(key)] Remove the value from the list (if it exists) 60

59 Chaining What is the Big O of each of these operations? Search(key) Insert(key,value) Remove(key) 61

60 Chaining The world s worst hash function int hash(t key) { return 0; } What would this hash function cause to happen if we are using a chaining approach? 62

61 Chaining What is the Big O of each of these operations? Search(key) Average: O(1), Worst Case: O(N) Insert(key,value) Average: O(1), Worst Case: O(1) Remove(key) Average: O(1), Worst Case: O(N) Operations depend on the average length of a chain (except for insert) 63

62 Chaining Other issues with Chaining Searching chains is still O(N) worst case Linear Search Potential Solutions Binary Search Trees Store a balanced binary search tree at each element in the Hash Table instead of a list Insertion becomes O(log(N)) Worst case lookup becomes O(log(N)) 64

63 Collision Handling The Problem If a malicious user knows what hash function you re using, they can intentionally cause your worst-case behavior Universal Hashing When the Hash Table is created, randomly choose a hash function independent of the keys that are going to be stored No single input gives worst-case behavior (just like randomized Quicksort) 66

64 Collision Handling Multi-Level Hashing Like chaining, but each element in the hash table holds another hash table with a different hash function By hashing multiple times, we can greatly decrease the odds of a collision Perfect Hashing If the set of possible keys is static (never changes), we can develop a perfect multi-level hash to give O(1) worst case performance e.g. The reserved keywords in a programming language are a static set of keys 67

65 Hash Tables Other Notes Hash Tables generally do provide a way for you to retrieve a list of the known keys Just keep in mind there is no guaranteed ordering of the keys C++ currently has no built-in hash table There s a proposal for unordered_map in the STL is on the table Google Sparse Hash provides C++ hash tables Boost C++ Libraries provides hash tables 68

66 CS102 Sets & Maps 1

67 Types of Containers Containers generally fall into 2 major categories Sequential Containers Elements are linearly arranged Each element has a position relative to the others Items are accessed by position Examples: vector, linked list, etc. Associative Containers Elements are not guaranteed to follow an ordering Designed for retrieval based on a unique identifier Items are accessed by key or value Examples: Hashtable, Binary Search Tree 3

68 Types of Containers Container Adapters Provide a specific & often restricted interface Rely on existing container to manage elements Interface is independent of underlying container Sequential Container Adapters Stack, Queue, Deque, Heaps, etc. Associative Container Adapters Set, MultiSet, Map, MultiMap, etc. 4

69 Types of Containers Sequential Container Adapters Stack, Queue, Deque We implemented them as linked lists Would they still work properly if we wanted to use arrays or vectors underneath? Would the performance suffer? Heap We implemented it using an array/vector Would it still work properly if we wanted to use a linked list underneath? Would the performance suffer? 5

70 Types of Containers Associative Container Adapters Set MultiSet (a.k.a. Bag) Map MultiMap Generally implemented using Hashtables or binary search trees (or variants) underneath 6

71 Sets Set An unordered collection of objects All elements in a set are unique (no duplicates) Each element is its own key Example Usage Given two lists of words, return a single list containing only the words that exist in both lists 7

72 Set Example Sets Initial set S = { 1, 3, 5, 8, 16, 22 } New values are added as normal Add 18 S = { 1, 3, 5, 8, 16, 22, 18 } Add 0 S = { 1, 3, 5, 8, 16, 22, 18, 0 } Duplicate values are rejected Add 8 S = { 1, 3, 5, 8, 16, 22, 18, 0 } Add 16 S = { 1, 3, 5, 8, 16, 22, 18, 0 } 8

73 Set Operations Set Operations search(value) See if the specified value is in the Set insert(value) Add a new value to the Set Return a value indicating whether or not anything was actually added to the Set remove(value) Remove an item from the set 9

74 Set Operations Set Operations (cont...) Union Combine two sets removing all redundant elements {1, 3, 5, 7} {2, 3, 4, 5} = {1, 2, 3, 4, 5, 7} Intersection Combine two sets keeping all shared elements {1, 3, 5, 7} {2, 3, 4, 5} = {3, 5} Difference Remove the elements in one set from another set (order matters!) {1, 3, 5, 7} - {2, 3, 4, 5} = {1, 7} {2, 3, 4, 5} - {1, 3, 5, 7} = {2, 4} 10

75 Sets Other Notes Set elements must be comparable Must be able to determine if something is already present in the set Either overload operators or pass comparison functions Sets are iterable Although sets have no specific ordering, you can still iterate over them Sets are not subscriptable You cannot index directly into a set like you would a vector (e.g. set[5] is not valid syntax) 11

76 Sets Given what you know, how would you go about implementing a Set? 12

77 Sets Given what you know, how would you go about implementing a Set? Binary Search Tree variants or Hashtables are both acceptable answers Many languages will give you both options and use names like TreeSet or HashSet to differentiate A TreeSet will generally be ordered while a HashSet will not C++ STL implements Sets with Red/Black Trees This means that the C++ STL set is actually an ordered set 13

78 MultiSet MultiSet An unordered collection of objects Duplicates are allowed Each element is its own key Works just like a Set, but allows repeated values Also known as a Bag Example Usage For each word that exists in a document, store the number of occurrences of that word 14

79 MultiSet Example MultiSet Initial set S = { 1, 3, 5, 8, 16, 22 } New values are added as normal Add 18 S = { 1, 3, 5, 8, 16, 22, 18 } Add 0 S = { 1, 3, 5, 8, 16, 22, 18, 0 } Duplicate values are added as normal Add 8 S = { 1, 3, 5, 8, 16, 22, 18, 0, 8 } Add 16 S = { 1, 3, 5, 8, 16, 22, 18, 0, 8, 16 } 15

80 MultiSet Operations MultiSet Operations search(value) See if the specified value is in the MultiSet Potentially also return the number of occurrences of a value in the MultiSet insert(value) Add a new value to the MultiSet (always succeeds) Duplicates are allowed remove(value) Remove all occurrences of the specified item from the MultiSet N inserts can be undone by 1 remove 16

81 MultiSets Given what you know, how would you go about implementing a MultiSet? 17

82 MultiSets Given what you know, how would you go about implementing a MultiSet? Binary Search Trees that allow duplicates Option 1: Allow duplicate elements in the tree Option 2: Each node in the tree contains a counter Hashtables where the key is the element and the value is a counter of how many of it exist Many other options Many languages will give you many options and use names like TreeMultiSet, HashMultiSet, TreeBag or HashBag to differentiate C++ STL implements MultiSets with Red/Black Trees 18

83 Map Map A Set of pairs of the form (key,value) The key must be unique The value can be non-unique Each key maps to exactly one value Each value can be mapped to by one or more keys Example Usage A dictionary. A word can be used as a key and its value could hold the word s definition, etymology, etc. 19

84 Map Example Maps M = { (32,John), (22,Jane), (40,Joe), (56,Jeff), (35,Joe) } Keys John Jane Joe Jeff Values 20

85 Map Operations Map Operations search(key) See if the specified key has an existing mapping insert(key,value) Add a new mapping from key to value If key already exists, reject the insert remove(key) Remove the value identified by key (and remove the key as well) Be careful if a value is pointed to by more than one key 21

86 Maps Other Notes Map keys must be comparable Must be able to determine if something is already present in the map Maps are iterable Although maps have no specific ordering, you can still iterate over them Most languages let you iterate over the set of keys or over the set of values 22

87 Maps Given what you know, how would you go about implementing a Map? 23

88 Maps Given what you know, how would you go about implementing a Map? Binary Search Tree variants or Hashtables are both acceptable answers Many languages will give you both options and use names like TreeMap or HashMap to differentiate A TreeMap will generally be ordered while a HashMap will not C++ STL implements Maps with Red/Black Trees This means that the C++ STL map is actually an ordered set 24

89 MultiMap MultiMap A Set of pairs of the form (key,value) Works just like a Map, but allows repeated keys Each key maps to exactly one or more values Each value can be mapped to by one or more keys Example Usage For each word that exists in a document, store the line number in the document for each occurrence of that word. 25

90 MultiMap Example MultiMap M = { (32,John), (22,Jane), (22,Joe), (56,Jeff), (35,Joe) } Keys John Jane Joe Jeff Values 26

91 MultiMap Operations MultiMap Operations search(key) See if the specified key has any existing mappings and return them all insert(key,value) Add a new mapping from key to value It does not matter if key already has a mapping remove(key) Remove all the values identified by key (and remove the key as well) Be careful if a value is pointed to by more than one key 27

92 MultiMap Given what you know, how would you go about implementing a MultiMap? 28

93 MultiMap Given what you know, how would you go about implementing a MultiMap? Binary Search Trees Option 1: Allow duplicate elements in the tree Option 2: Each node in the tree contains a list Hashtables where the key is the element and the value is a list of mapped values Many other options C++ STL implements MultiMaps with Red/Black Trees 29

94 The C++ STL "map" Class 3