CSE 214 Computer Science II Searching

Transcription

1 CSE 214 Computer Science II Searching Fall 2017 Stony Brook University Instructor: Shebuti Rayana

2 Introduction Searching in a list of values is a common computational task Some real-world applications: Retrieve a student record, bank account record, credit record etc. Searching words in a dictionary Searching algorithms that we will cover: Ø Serial Search Ø Binary Search Ø Search by Hashing Shebuti Rayana (CS, Stony Brook University) 2

3 Problem: Searching We are given a array/list of records. Each record has an associated key. Give efficient algorithm for searching for a record containing a particular key. Efficiency is quantified in terms of time analysis (number of comparisons) to retrieve an item. Shebuti Rayana (CS, Stony Brook University) 3

4 Searching: Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 700 ] Number Number Number Number Number Number Each record in list has an associated key. In this example, the keys are ID numbers. Number Given a particular key, how can we efficiently retrieve the record from the list? Shebuti Rayana (CS, Stony Brook University) 4

5 Tables A table, or dictionary, is an abstract data type whose data items are stored and retrieved according to a key value. The items are called records. Each record can have a number of data fields. The records are ordered based on one of the data fields, named the key field. The record we are searching for has a key value that is called the target. The table may be implemented using a variety of data structures: array, tree, heap etc. Shebuti Rayana (CS, Stony Brook University) 5

6 Serial Search Also called Linear or sequential search 1. Step through array/list of records, one at a time. 2. Look for record with matching key. 3. Search stops when record with matching key is found or when search has examined all records without success. Shebuti Rayana (CS, Stony Brook University) 6

7 Serial Search: Implementation Searching key in an Array public static int search(int[] a, int target) { } int i = 0; boolean found = false; while ((i < a.length) &&! found) { } if (a[i] == target) else i++; found = true; if (found) return i; else return 1; Shebuti Rayana (CS, Stony Brook University) 7

8 Serial Search: Implementation Searching key in a Table public static int search(tableclass[] a, int target) { } int i = 0; boolean found = false; while ((i < a.length) &&!found){ } if (a[i].getkey() == target) else i++; found = true; if (found) return i; else return 1; With this code you can do serial search in any sequential data structure Shebuti Rayana (CS, Stony Brook University) 8

9 Complexity Analysis: Serial Search on N elements Best Case: The best case time requires just one comparison: O(1) When the item is in the front of the array Average Case: The average-case time requires (N+1)/2 comparisons: O(N) In average case consider all the possibilities and take the average ( N)/N = (N(N+1)/2)/N = (N+1)/2 Worst Case: The worst-case time requires N comparisons: O(N) When the item is not present or found at the end of the array Shebuti Rayana (CS, Stony Brook University) 9

10 Although serial search is a linear searching algorithm, do you think this is the most efficient algorithm? Shebuti Rayana (CS, Stony Brook University) 10

11 Binary Search A faster algorithm than serial search Can be applied to any random-access data structure where the data elements are sorted. Additional parameters: first: index of the first element to examine size: number of elements to search starting from the first element above Shebuti Rayana (CS, Stony Brook University) 11

12 Some examples where binary search is applicable Searching an array of integers in which the array is sorted from the smallest integer (at the front) to the largest integer (at the end). Searching an array of strings in which the strings are sorted alphabetically. Searching an array of Objects containing information about some item, such as an auto part, and the array is sorted by part numbers from the smallest to largest. Shebuti Rayana (CS, Stony Brook University) 12

13 Binary Search Precondition: If size > 0, then the array has size elements starting with the element at the first index. In addition, these elements are sorted. Postcondition: If target appears, the position of the target is returned (a non-negative integer). Otherwise, -1 is returned. Shebuti Rayana (CS, Stony Brook University) 13

14 Recursive Binary Search [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] Look at the middle element of the array first. If it is the target, you are done If not, it might be in the left half of the array or the right half, but not both! Shebuti Rayana (CS, Stony Brook University) 14

15 Recursive Binary Search public static int search(int[] a, int first, int size, int target) { } int mid; if (size <= 0) return 1; else { mid = first + (size/2); if (target == a[mid]) return mid; else if (target < a[mid]) return search(a, first, size/2, target); else return search(a, mid+1,(size-1)/2, target); } Shebuti Rayana (CS, Stony Brook University) 15

16 Recursive Binary Search : Example a [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Function call: search(a, 0, 10, 42) mid = first + size/2 = /2 = 5 and a[5] = 52 which is greater than 42. So the algorithms makes the next recursive call search(a, 0, 5, 42) a [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Shebuti Rayana (CS, Stony Brook University) 16

17 Recursive Binary Search : Example Each recursive call throws away at least half of the elements. a [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] mid = first + size/2 = 0 + 5/2 = 2 and a[2] = 29 which is smaller than 42. So the algorithms makes the next recursive call search(a, 3, 2, 42) a [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Now, mid = 3 + 2/2 = 4 and a[4] = 42, target is found and index 4 is returned Shebuti Rayana (CS, Stony Brook University) 17

18 Recursive Binary Search in a Table public static int search(tableclass[] a, int first, int size, int target) { int mid; if (size <= 0) return 1; else { mid = first + (size/2); if(target == a[mid].getkey()) return mid; else if (target < a[mid].getkey()) return search(a, first, size/2, target); else return search(a,mid+1,(size-1)/2,target); } } Shebuti Rayana (CS, Stony Brook University) 18

19 Complexity Analysis: Binary Search on N elements Let T(N) = the total number of comparisons for a search on N elements. T(N) = 1 + T(N/2) T(N/2) = 1 + T(N/4)... T(1) = 1 Ø T(N) = = O(log 2 N) [worst-case time] Example: Search 42 a Is Binary Search in a sorted array better than searching in BST? [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] T(10) = 1 + T(5) = T(2) = = 3 ~ log 2 (10) Shebuti Rayana (CS, Stony Brook University) 19

20 What is the limitation of binary search? Shebuti Rayana (CS, Stony Brook University) 20

21 Hashing Data records are stored in a hash table. The position of a data record in the hash table is determined by its key. A hash function maps keys to positions in the hash table. If a hash function maps two keys to the same position in the hash table, then a collision occurs. Shebuti Rayana (CS, Stony Brook University) 21

22 Goals of Hashing An insert without a collision takes O(1) time. A search also takes O(1) time, if the record is stored in its proper location. The hash function can take many forms: If the key k is an integer: k % tablesize If key k is a String : k.hashcode() % tablesize Any function that maps k to a table position! The table size should be a prime number. To guarantee even distribution and minimize collision Shebuti Rayana (CS, Stony Brook University) 22

23 A Simple Hashing: Example Let the hash table be an 11-element array. If k is the key of a data record, let H(k) represent the hash function, where H(k) = k mod 11. Insert the keys 83, 14, 29, 70, 55, 72: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] Inserting 72 now would cause a collision! Shebuti Rayana (CS, Stony Brook University) 23

24 Collision Resolution Techniques Open-addressing Linear Probing Quadratic Probing Double Hashing Chained hashing (or Chaining) Shebuti Rayana (CS, Stony Brook University) 24

25 Open-Addressing: Linear Probing During insert of a key k into position p: If position p already contains a different key, then examine positions p + 1, p + 2, etc.* until an empty position is found and insert k there. During a search for key k at position p: If position p already contains a different key, then examine positions p + 1, p + 2, etc.* until either the key is found or an unused position is encountered. *wrap around to beginning of array if p + i > tablesize Shebuti Rayana (CS, Stony Brook University) 25

26 Open-Addressing: Linear Probing Example: Insert additional key 72, 36, 48 using H(k) = k mod 11 and linear probing [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] Shebuti Rayana (CS, Stony Brook University) 26

27 Open-Addressing: Quadratic Probing If position p already contains a different key, then examine positions p + 1, p + 4, p + 9, etc. until either the key is found or an empty position is encountered. Example: Insert additional key 22, 66, 77 using H(k) = k mod 11 and quadratic probing [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] Quadratic probing over Linear probing? In quadrating probing values are more spread out which reduces collision. Shebuti Rayana (CS, Stony Brook University) 27

28 Problem in Probing If we remove a key from the hash table, can we get into problems? [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] Remove 83. Now search for 48. We can t find 48 due to the gap in position 6! Shebuti Rayana (CS, Stony Brook University) 28

29 Problem in Probing: Solution Add another array with boolean values that indicate whether the position is currently used or has been used in the past. If a key is removed, leave the boolean value set at true so we can search past it if necessary [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] T F F T T T T T T T F Shebuti Rayana (CS, Stony Brook University) 29

30 Load Factor A critical influence on performance of an open addressing hash table is the load factor The load factor α of a hash table is given by the following formula: α = (number of elements in table)/(size of table) Thus, 0 < α < 1 for linear probing. (α can be greater than 1 for other collision resolution methods like chained hashing) For linear probing, as α approaches 1, the number of collisions increases Shebuti Rayana (CS, Stony Brook University) 30

31 Hash Table ADT with Linear Probing Invariants for the Hash Table: The number of elements in the table manyitems We try to store an element with a given key k at location hash(k). If a collision occurs, linear probing is used to find a location to store the element and its associated key. When an open address is found at index i, then the element is placed in data[i], and its key is placed in key[i]. If index i has never been used in the table, data[i] and key[i] are set to null. If index i is or has been used in the past, then hasbeenused[i] is true; otherwise it is false. Shebuti Rayana (CS, Stony Brook University) 31

32 Various Hash functions Division hash function convert key to a positive integer return the integer modulo the table size Mid-square hash function convert key to an integer multiply the integer by itself return several digits in the middle of result Multiplicative hash function convert the key to an integer multiply the integer by a constant less than 1 return the first several digits of the fractional part Shebuti Rayana (CS, Stony Brook University) 32

33 Clustering Problem in Probing Linear Probing can cause significant primary clustering problem. Quadratic probing can also cause secondary clustering problem. Shebuti Rayana (CS, Stony Brook University) 33

34 Primary Clustering Primary Clustering is the tendency for a collision resolution scheme such as linear probing to create long runs of filled slots near the hash position of keys. If the primary hash index is x, subsequent probes go to x+1, x+2, x+3 and so on, this results in Primary Clustering. Once the primary cluster forms, the bigger the cluster gets, the faster it grows. And it reduces the performance. Shebuti Rayana (CS, Stony Brook University) 34

35 Secondary Clustering Secondary Clustering is the tendency for a collision resolution scheme such as quadratic probing to create long runs of filled slots away from the hash position of keys. If the primary hash index is x, probes go to x+1, x+4, x+9, x+16, x+25 and so on, this results in Secondary Clustering. Secondary clustering is less severe in terms of performance hit than primary clustering, and is an attempt to keep clusters from forming by using Quadratic Probing. The idea is to probe more widely separated cells, instead of those adjacent to the primary hash site. Shebuti Rayana (CS, Stony Brook University) 35

36 Reducing Clustering: Double Hashing To reduce clustering, use double hashing. Define two hash functions: hash1 and hash2 Use the hash1 function to determine the initial location of the key in the hash table. If a collision occurs, use the hash2 function to determine how far to move ahead to look for a vacant location in the hash table. Shebuti Rayana (CS, Stony Brook University) 36

37 Double Hashing In double hashing, the array index must not leave the range [0, tablesize-1] To keep the index within this range, we do following Suppose that i is the index we have just examined (with a collision) then the next index to examine is (i + hash2(key)) % tablesize As we step through the array, we must mane sure every position is examined. Problem: with double hashing, we could come back to our starting point before we examine all the positions. Shebuti Rayana (CS, Stony Brook University) 37

38 Double Hashing Make sure the tablesize is relatively prime with the value returned by hash2. Choose tablesize as a prime number Have hash2 return values in the range [1, tablesize-1] Double hashing rule by Donald Knuth: 1. Both tablesize and (tablesize 2) should be prime numbers. For example, 1231 is a prime and so is 1229 (two such primes separated by 2 are called twin primes) 2. hash1(key) = key % tablesize 3. hash2(key) = 1 + (key % (tablesize - 2)) Shebuti Rayana (CS, Stony Brook University) 38

39 Example: Double Hashing H1(k) = k % 1231, [tablesize = 1231] H2(k) = 1 + (k % 1229) For key k = 2000, H1(k) = 769 If location 769 is occupied, H2(k) = 772 So we check position ( ) % 1231 = 310 to see if it is occupied. If it is, we check position ( ) % 1231 = 1082, etc. d = gcd(h2(k), tablesize) = 1 The table size is relatively prime to the value returned by the second hash function H2. Shebuti Rayana (CS, Stony Brook University) 39

40 Example: Double Hashing (cont.) H1(k) = k % 100 H2(k) = 2 + (k % 52) For key k = 204, H1(k) = 4 If location 4 is occupied, H2(k) = 50 So we check position (4+50) % 100 = 54 to see if it is occupied. We came back to the starting point If it is, we check position (54+50) % 100 = 4. Table size is not relatively prime to the value returned by the second hash function hash2. d = gcd(h2(k), tablesize) = 50 Shebuti Rayana (CS, Stony Brook University) 40

41 Chained Hashing The maximum number of elements that can be stored in a hash table implemented using an array is the tablesize (α = 1.0). We can have a load factor greater than 1.0 by using chained hashing. Each array position in the hash table is a head reference to a linked list of keys. All colliding keys that hash to an array position are inserted to that linked list. Insertion: O(1), Deletion: O(1), if doubly linked list is used Shebuti Rayana (CS, Stony Brook University) 41

42 Example: Chained Hashing Shebuti Rayana (CS, Stony Brook University) 42

43 Average Search Time The average number of table elements examined in a successful search is approximately: (1 + 1 / (1 - α)) / 2 using linear probing* -ln(1 - α) / α using double hashing* 1 + α / 2 using chained hashing *assuming a non-full hash table Shebuti Rayana (CS, Stony Brook University) 43

44 Average Search Time: Chaining Suppose, Table size = m and number of elements = n Search finds the element x $ Let s call all the elements x %, x ', x (,, x * On the average, how many of the remaining n i elements map to the same list as x $? y = / 1 m 23$4% [Because the probability of each element getting mapped to the same slot is 1 m.] Therefore, on average, the number of elements examined before finding x $ is 1 + y. * Shebuti Rayana (CS, Stony Brook University) 44

45 Average Search Time: Chaining * y = / 1 m 23$4% * = 1 m / 1 23$4% = (*8$) : Therefore, on average, the number of elements examined before finding x $ is 1 + (n i) m. The last step is to take the average of this formula over all x $. Shebuti Rayana (CS, Stony Brook University) 45

46 Average Search Time: Chaining Thus, on average, the number of searches to find any element is the average over all x i. Which is (1 n) X number of examinations to find x i [over all x i ] = % * (1 + *8$ ) * $3% = % (n + * *8$ : * $3% ) : % = 1 + * (n i) :* $3% = 1 + *8% = 1 + % α % ': ' ': Main target is that the load factor stays constant and both m and n keep increasing. That is, find the limit of the above expression as m, n Shebuti Rayana (CS, Stony Brook University) 46

47 Average number of table elements examined during successful search Load factor(a ) Open addressing, linear probing ½ (1+1/(1-a )) Open addressing double hashing -ln(1-a )/a Chained hashing 1+a / Not applicable Not applicable Not applicable Not applicable Not applicable Not applicable 2.50 Shebuti Rayana (CS, Stony Brook University) 47

48 Average number of table elements examined during successful search Shebuti Rayana (CS, Stony Brook University) 48