Implementing Algorithms

Transcription

1 Implementing Algorithms 1 Data Structures implementing algorithms arrays and linked lists 2 Implementing the Gale-Shapley algorithm selecting data structures overview of the selected data structures 3 Graphs definition and terminology adjacency matrices graph traversals: breadth first and depth first CS 401/MCS 401 Lecture 3 Computer Algorithms I Jan Verschelde, 22 June 2018 Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

2 Implementing Algorithms; Graphs 1 Data Structures implementing algorithms arrays and linked lists 2 Implementing the Gale-Shapley algorithm selecting data structures overview of the selected data structures 3 Graphs definition and terminology adjacency matrices graph traversals: breadth first and depth first Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

3 implementing algorithms Niklaus Wirth: algorithms + data structures = programs. What do we mean by implementing an algorithm? Definition Implementing an algorithm is selecting the best data structures that will make the algorithm run efficiently. Example: we proved that in the worst case, the Gale-Shapley algorithm for n men and n women does require no more than n 2 iterations of the loop. This proof does not imply that the running time of the Gale-Shapley algorithm is O(n 2 ) because we have not proven yet that every step in each iteration of the loop has constant time. We denote constant time by O(1). Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

4 Implementing Algorithms; Graphs 1 Data Structures implementing algorithms arrays and linked lists 2 Implementing the Gale-Shapley algorithm selecting data structures overview of the selected data structures 3 Graphs definition and terminology adjacency matrices graph traversals: breadth first and depth first Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

5 arrays An array A is a sequence of consecutive data elements of fixed size, size(a), and the cost of accessing the i-th element in A is O(1). Example: an array which stores 3,, 4: 3 4 Exercise 1: Given is an array of numbers A, sorted in increasing order, and some number x. Prove that finding the position of x in A can be done in sublinear time, in the size of A. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June 2018 / 43

6 two dimensional arrays A matrix is stored as an array of arrays: A A = Double indexing works as follows: A 2,1 = = A[2][1] = A[2, 1] Given indices i and j, the cost of accessing A i,j = A[i, j] is O(1). Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

7 linked lists Example: a linked list L to store 3,, 4: L Node Node Node NULL next: data: 3 next: data: next: data: 4 A node in a linked list consists of a data element and a pointer. A linked list consists of a head node, linked to the next node, which is linked to the next node, etc, linked to the last node in the list, which has NULL as the pointer to its next node. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

8 circular lists In a linked list, one often needs a separate pointer to the last element of the list, to append an element to the end of the list. When the order among the stored data does not matter, distinguishing between first and last does not matter either, and the list is a pointer to some element in the list. Example: a circular linked list L stores 3,, 4: L Node next: data: 3 Node next: data: Node next: data: 4 Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

9 doubly linked lists Example: A doubly linked list L to store 3,, 4: L Node next: prev: data: 3 Node Node next: prev: data: next: prev: data: 4 NULL NULL At the expense of one extra pointer per node, reverse transversal becomes efficient. Doubly linked list can also made to be circular. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

10 properties of linked lists The cost to locate an element in a list of n elements is O(n), because to reach the i-th element in a list, all previous i 1 nodes need to be visited. Linked lists are dynamic data structures: Insertion of a new node costs O(1) time. Given the pointer to a node in a doubly linked list, the deletion of the node costs O(1) time. Exercise 2: Given is a doubly linked list with n elements. Prove the above statements: inserting and deleting a node is O(1), independent of the size n of the list. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

11 Implementing Algorithms; Graphs 1 Data Structures implementing algorithms arrays and linked lists 2 Implementing the Gale-Shapley algorithm selecting data structures overview of the selected data structures 3 Graphs definition and terminology adjacency matrices graph traversals: breadth first and depth first Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

12 input data of the Gale-Shapley algorithm Input data: every man and woman has a preference list with no ties. Example: Consider 4 men {1, 2, 3, 4} and 4 women {a, b, c, d}. Preferences for men Preferences for women 1 : a c b d a : : b a d c b : : a c b d c : : d a c b d : Read the first line as 1 prefers a to c, 1 prefers c to b, 1 prefers b to d, a prefers 2 to 3, a prefers 3 to 4, a prefers 4 to 1. The number of men and women is fixed at the start of the algorithm. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

13 storing the input data Order the n men and n women into a sequence 1, 2,..., n. Because of the fixed size of constants, we use a two dimensional array: Preferences for men 1 : : : : ManPref For a man m: ManPref[m, i] stores the i-th woman on the preference list of m. Similarly for the women, the two dimensional array WomanPref stores their preferences. For a woman w: WomanPref[w, i] stores the i-th man on the preference list of w. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

14 the first statement in the Gale-Shapley algorithm Consider the first statement in the Gale-Shapley algorithm: 1 Let m M: isfree(m) and there is a w W : hasproposed(m, w) is false. We need to be able to locate the next free man. The set of free men changes: A man is no longer free if his proposal is accepted. A man becomes free if an engagement is broken. A linked list FreeMen will store the set of free men: The next free men m is at the front of the list: O(1). If his proposal is accepted, then we delete the first element: O(1). If the woman who accepted the proposal of m breaks up with m, then m is inserted to the front of the list of free men: O(1). However, it takes O(n) time to initialize FreeMen. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

15 proposing to the highest ranked Consider the second statement in the Gale-Shapley algorithm: 2 Let w W : w is the highest ranked in the preference list of m for whom hasproposed(m, w) is false. Observe: 1 Proposals are made in the order of preference. 2 Preferences are stored in decreasing order: highest ranked first. ManPref m next[m] Consider the preferences of m. Assume m proposed to 2 and 1. Next time m is free, he proposes to 4. next is an array which stores for each m his next highest ranked: m proposes to ManPref[m, next[m]] Initially, next[m] = 1 for all m. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

16 storing the engagements Consider the beginning of the third statement in the loop: 3 hasproposed(m, w) := true If isfree(w) then S := S (m, w) The hasproposed(m, w) := true is implemented by incrementing next[m], as: next[m] := next[m] + 1. Need to know: isfree(w) and if not, who is w engaged to? Introduce an array current of size n: If isfree(w), then current[w] = 0. If not isfree(w), then current[w] = m, w is engaged to m. The current implements the set of engaged pairs S. Because accessing an array element is O(1), checking if w is free and to whom w is engaged is O(1). Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

17 ranking the men along preferences of the women Consider the breaking of an engagement: 3 if... else there is an m M: (m, w) S (w is engaged to m ) if w prefers m to m then S := S \ (m, w) (m and w are no longer engaged) If there is an m M: (m, w) S, then current[w] = m. How to determine if w prefers m to m? Let there be a two dimensional array Ranking such that Ranking[w, m] < Ranking[w, m ] if w prefers m to m. Ranking[w, m] is the position of m in the preference list of w, e.g.: if Ranking[w, m] = 2, then m is the second best choice of w. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

18 building the auxiliary data structure Ranking The cost of consulting Ranking[w, m] is O(1), but what about the cost of constructing Ranking? Preferences for women 1 : : : : WomanPref for all woman w = 1, 2,..., n: for all ranks r = 1, 2,..., n: Ranking[w, WomanPref[w, m]] := r The loop makes n 2 assignments, so the cost is O(n 2 ). Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

19 Implementing Algorithms; Graphs 1 Data Structures implementing algorithms arrays and linked lists 2 Implementing the Gale-Shapley algorithm selecting data structures overview of the selected data structures 3 Graphs definition and terminology adjacency matrices graph traversals: breadth first and depth first Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

20 overview of the selected data structures 1 ManPref is a two dimensional array for the preferences of n men. 2 WoManPref is a two dimensional array for the preferences of n women. 3 FreeMen is a linked list of all free men. 4 next is an array of size n for the next woman to propose to. current is an array of size n to store the engaged pairs. 6 Ranking is a two dimensional array to rank men. Exercise 3: The construction of Ranking is O(n 2 ). It is the most time consuming operation. In which scenario is Ranking not needed? Explain. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

21 running the Gale-Shapley algorithm once again Consider 4 men {1, 2, 3, 4} and 4 women {1, 2, 3, 4}. Preferences for men Preferences for women 1 : : : : : : : : Read the first line as man 1 prefers woman 1 to 3, prefers 3 to 2, prefers 2 to 4; woman 1 prefers man 2 to 3, prefers 3 to 4, prefers 4 to 1. Exercise 4: Run the Gale-Shapley algorithm on the above input data, show the evolution of all data structures in each step. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

22 the Gale-Shapley algorithm is an efficient algorithm Theorem For n men and n women, the running time of the Gale-Shapley algorithm is O(n 2 ). Proof. Consider the costs of the data structures defined above. We have two n-by-n arrays: ManPref and WomanPref, given on input. The other n-by-n array Ranking is constructed only once which takes O(n 2 ) time. Accessing arrays is O(1). The linked list FreeMen takes O(n) time to initialize. Both deleting a free man and inserting a free man is O(1). The initialization of the arrays next and current takes O(n) time. Accessing arrays is O(1). After initialization, all steps in the algorithm take O(1) time. As proven in Lecture 1, the number of iterations is bounded by n 2, so the Gale-Shapley algorithm runs in O(n 2 ) time. Q.E.D. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

23 Implementing Algorithms; Graphs 1 Data Structures implementing algorithms arrays and linked lists 2 Implementing the Gale-Shapley algorithm selecting data structures overview of the selected data structures 3 Graphs definition and terminology adjacency matrices graph traversals: breadth first and depth first Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

24 graphs, terminology and definition A node in a graph is called a vertex. A connection between two vertices is an edge. A graph G is defined by a tuple (V, E): V is the set of vertices, and E is the set of edges. An example: A B C D V = {A, B, C, D} E = {(A, B), (A, C), (A, D), (C, D), (D, D)} Two vertices are adjacent if there is an edge between them. A path is a sequence of successively adjacent vertices. For example, a path from B to D is B, A, D. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

25 directed and undirected graphs In an undirected graph (which is the default), the edge (A, B) means: there is an edge from A to B and there is an edge from B to A. A B V = {A, B, C, D} E = {(A, B), (A, C), (A, D), (C, D), (D, D)} C D A B C D V = {A, B, C, D} E = {(A, B), (B, A), (C, A), (A, D), (D, C), (D, D)} In a directed graph or digraph, (A, B) means there is an edge from A to B and (B, A) means there is an edge from B to A. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

26 weighted graphs In an weighted graph, a value (a weight) is associated to every edge. A B 8 3 C D 4 7 A B C D 4 7 Edges in a weighted graphs are triplets (u, v, w), with w the weight of the edge which connects u to v. A cycle is a path where the first vertex equals the last one. For example: A, D, C, A is a cycle. A graph without cycles can be viewed as a tree. The degree of a vertex is the number of edges that contain that vertex. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

27 Implementing Algorithms; Graphs 1 Data Structures implementing algorithms arrays and linked lists 2 Implementing the Gale-Shapley algorithm selecting data structures overview of the selected data structures 3 Graphs definition and terminology adjacency matrices graph traversals: breadth first and depth first Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

28 adjacency matrices For a graph G = (V, E), the adjacency matrix M has as many rows and columns as the number of vertices in V ; for all v i, v j V : if (v i, v j ) E: M i,j = 0; for all v i, v j V : if (v i, v j ) E: M i,j = 1. A B C D A B C D A B C D A B C D A B C D A B C D Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

29 adjacency matrices for weighted graphs For a weighted graph G = (V, E), the adjacency matrix M has as many rows and columns as the number of vertices in V ; for all v i, v j V : if (v i, v j ) E: M i,j = 0; for all v i, v j V : if (v i, v j ) E: M i,j = w, w is the weight of (v i, v j ). A B 8 3 C D 4 7 A B C D A B C D A B C D 4 7 A B C D A B C D Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

30 Implementing Algorithms; Graphs 1 Data Structures implementing algorithms arrays and linked lists 2 Implementing the Gale-Shapley algorithm selecting data structures overview of the selected data structures 3 Graphs definition and terminology adjacency matrices graph traversals: breadth first and depth first Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

31 visiting each vertex in a graph Consider the graph: Problem: visit each vertex in a systematic order. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

32 breadth-first search We start at vertex at 0: visit 1, 3, at 1: visit and 6 Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

33 breadth-first search continued at 3: visit 2 and 8 Every vertex is visited only once. Although there is an edge from 4 to, the breadth-first search will not visit this edge because has already been visited. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

34 breadth-first search continued at 4: visit 7 Observe that the graph on the right is a tree. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

35 the breadth-first tree At the left is the result of the breadth-first search. At the right is the breadth-first tree The breadth-first tree of a graph contains all the vertices of the graph and the edges visited in the breadth-first search. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

36 the breadth-first search algorithm Input: G = (V, E). Select the start vertex v V. Initialize the queue Q with v: Q.push(v). while not Q.empty() do u = Q.pop(); visit u; for all v adjacent to u do if v is not visited then if v Q then Q.push(v). Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

37 running the breadth-first search algorithm V = {0, 1, 2, 3, 4,, 6, 7, 8} E = {(0, 1), (0, 3), (0, 4), (1, ), (1, 6), (2, 3), (2, ), (3, 8), (4, ), (4, 6), (4, 7), (, 7), (, 8), (7, 8)} Q = 0 pop 0, visit 0, Q = 1, 3, 4 pop 1, visit 1, Q = 3, 4,, 6 pop 3, visit 3, Q = 4,, 6, 2, 8 pop 4, visit 4, Q =, 6, 2, 8, 7 pop, visit, Q = 6, 2, 8, 7 pop 6, visit 6, Q = 2, 8, 7 pop 2, visit 2, Q = 8, 7 pop 8, visit 8, Q = 8 pop 7, visit 7, Q = Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

38 depth-first search We start at vertex Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

39 depth-first search continued Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

40 depth-first search continued Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

41 the depth-first search tree The depth-first search tree of a graph contains all the vertices of the graph and the edges visited in the depth-first search. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

42 the depth-first search algorithm algorithm VISIT(V, E, u, Visited) // Visits all vertices in the graph (V, E), starting at u. // Visited collects the vertices visited. visit u; Visited.push(u); for all v V, v is adjacent to u do if v Visited then VISIT(V, E, v, Visited). We call algorithm VISIT with vertex 0 for u and an empty list for Visited. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43

43 running the depth-first search algorithm V = {0, 1, 2, 3, 4,, 6, 7, 8} E = {(0, 1), (0, 3), (0, 4), (1, ), (1, 6), (2, 3), (2, ), (3, 8), (4, ), (4, 6), (4, 7), (, 7), (, 8), (7, 8)} visit 0, VISIT(V, E, 1, {0}) visit 1, VISIT(V, E,, {0, 1}) visit, VISIT(V, E, 2, {0, 1, }) visit 2, VISIT(V, E, 3, {0, 1,, 2}) visit 3, VISIT(V, E, 8, {0, 1,, 2, 3}) visit 8, VISIT(V, E, 7, {0, 1,, 2, 3, 8}) visit 7, VISIT(V, E, 4, {0, 1,, 2, 3, 8, 7}) visit 4, Visited = {0, 1,, 2, 3, 8, 7, 4} visit 6, Visited = {0, 1,, 2, 3, 8, 7, 4, 6} The tree of recursive calls is the depth-first search tree. Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June / 43