Xoo. Address. October 19,

Transcription

1 Xoo October 19, Address

2

3 3

4

5 Preface :P 1 Copyright 2004 lightzju@hotmail.com. All rights reserved. lightzju@hotmail.com 2004 Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with the Invariant Sections being Contributors, no Front-Cover Texts, and no Back-Cover Texts. This document is distributed in the hope that it will be useful,but without any warranty. GNU 1.2 Invariant Sections Contributors Front-Cover Texts Back-Cover Texts 1 L A TEX 2ε

6

7 Contents Contents Part I Foundations 1 The Role of Algorithms in Computing Getting Started Insertion sort Analyzing algorithms Designing algorithms Problems Growth of Functions Asymptotic notation Standard notations and common fuctions Recurrences The substitution method The recursion-tree method The master method Problems Probabilistic Analysis and Randomized Algorithms The hiring problem Indicator random variables Randomized algorithms Part II Sorting and Order Statistics

8 8 CONTENTS 6 Heapsort Heaps Maintaining the heap property Building a heap The heapsort algorithm Priority queues Problems The C implementations of algorithms Quicksort Description of quicksort Performance of quicksort A randomized version of quicksort Analysis of quicksort Problems The C implementations of algorithms Sorting in Linear Time Lower bounds for sorting Counting sort Radix sort The C implementations of algorithms Medians and Order Statistics Minimum and maximum Selection in expected linear time Selection in worst-case linear time Problems Part III Data Structures 10 Elementary Data Structures Stacks and queues Linked lists Implementing pointers and objects Representing rooted trees Problems The C implementations of algorithms Hash Tables Direct-address tables Hash tables Hash fuctions Open addressing

9 CONTENTS 9 12 Binary Search Trees What is a binary search tree? Querying a binary search tree Insertion and deletion Problems Red-Black Trees Properties of red-black trees Rotations Insertion Deletion Problems The C implementations of algorithms Augmenting Data Structures Dynamic order statistics How to augment a data structure Interval trees Part IV Advanced design and Analysis Techniques 15 Dynamic Programming Assembly-line scheduling Matrix-chain multiplication Elements of dynamic programming Longest common subsequence Optimal binary search trees Problems Greedy Algorithms An activity-selection problem Elements of the greedy strategy Index Part One

10

11 Part I Foundations

12

13 1 The Role of Algorithms in Computing Sorry,Nothing yet!

14

15 2 Getting Started K.I.S.S keep it simple stupid 2.1 Insertion sort Sorry,Nothing yet! 2.2 Analyzing algorithms Exercise Selection-Sort(A) 1 for j 1 to length[a] 2 do index j 3 i j while i length[a] 5 do if A[index] > A[i] 6 then index i 7 i i A[j] A[index] Θ(n 2 ) 2.3 Designing algorithms Exercise Merge

16 16 2 Getting Started Merge(A, p, q, r) 1 n1 q p n2 r q 3 create arrays L[1.. n 1 ] and R[1.. n 2 ] 4 for i 1 to n 1 5 do L[i] A[p + i 1] 6 for j 1 to n 2 7 do R[j] A[q + j] 8 i 1 9 j 1 10 for k p to r 11 do if i > n 1 12 then A[k] R[j] 13 j j if j > n 2 15 then A[k] L[i] 16 i i if i n 1 and j n 2 18 then if L[i] R[j] 19 then A[k] L[i] 20 i i else A[k] R[i] 22 j j + 1 Exercise Insertion-Sort Insertion(A, p, r) 1 key A[r] 2 i r 1 3 while A[i] > key and i > p 1 4 do A[i + 1] A[i] 5 i i 1 6 A[i + 1] key Insertion-Sort(A, p, r) 1 if p < r 2 then q r 1 3 Insertion-Sort(A, p, q) 4 Insertion(A, p, r) Exercise Binary-Search

17 Binary-Search(A, key, l, r) 2.4 Problems 17 1 if l > r 2 then return error 3 else i l+r 2 4 if l < r 5 then if A[i] > key 6 then return Binary-Search(A, key, l, i 1) 7 else if A[i] < key 8 then return Binary-Search(A, key, i + 1, r) 9 else return i 10 else if A[l] = key 11 then return l 12 else return nil { c if n = 1 T(n) = T(n/2) + c if n > 1 2 n : n, n 2, n 4,..., 1 c, } c,.{{.., c, c} T(n) = c lg n + c Θ(lg n) lg n Problems Problem 2-1 Insertion sort on small arrays in merge sort Merge-Insertion-Sort(A, p, r, k) 1 if r p + 1 k 2 then Insertion-Sort(A, p, r) 3 else q (r + p)/2 4 Merge-Insertion-Sort(A, p, q, k) 5 Merge-Insertion-Sort(A, q + 1, r, k) 6 Merge(A, p, q, r) Merge-Sort k Insertion-Sort a. k Θ(k 2 ) n/k Θ(nk) b. n Merge-Sort Θ(n lg(n)) { ck 2 if n k T(n) = 2T(n/2) + cn if n > k

18 18 2 Getting Started cn cn cn/2 cn/2 cn cn/4 cn/4 cn/4 cn/4 cn. ck } 2, ck 2,{{, ck 2 } n/k. cnk T(n) = cn lg(n/k) + cnk Θ(n lg(n/k)) c. Merge-Sort c 1,Insertion-Sort c 2 c 1 > c 2 T 1 (n) = c 1 n lg n + c 1 n (2.1) T 2 (n) = c 1 n lg(n/k) + c 2 nk (2.2) T 1 (n) > T 2 (n) (2.3) c 2 k < 1 + lg k c 1 (2.4) k 1 d. Problem 2-2 Correctness of bubblesort a. b. A[i] A[i.. n] A[j] A[j.. n] Initialization j = n A[n] A[n] Maintenance A[j 1] A[j 1.. n] j j 1 loop invariant Termination j = i A[i] A[i.. n] c. A[1] A[2] A[n] Loop invariant: A[i] A[i 1] Initialization i = 1 A[1] 2 Maintenance A[i.. n] A[i] i i + 1, A[i + 1] A[i] loop invariant A[0] =

19 2.4 Problems 19 Termination i = n + 1, A[1] A[2] A[n] 3 d. Bubblesort(A) 1 for i 1 to length[a] 2 do for j length[a] downto i do if A[j] > A[j 1] 4 then exchange A[j] A[j 1] cost times c 1 n + 1 n c 2 1 t i n c 3 1 (t i 1) n c 4 1 (t i 1) T(n) = c 1 (n + 1) + c 2 (n + (n 1) + + 1) + c 3 ((n 1) + + (n 2) + + 2) + c 3 ((n 1) + (n 2) + + 2) ( ) n(n + 1) n(n 1) = c 1 (n + 1) + c 2 + (c 3 + c 4 ) = c ( 2 + c 3 + c 4 n 2 + c 1 + c c ) 3 + c 4 n + (c 1 c 3 c 4 ) 2 Θ(n 2 ) Θ(n 2 ), Problem 2-3 Correctness of Horner s rule a. Θ(n) b. Naive-Polynomial-Evaluate 1 y a 0 2 e 1 3 for i 1 to n 4 do e e x 5 y y + a i e 6 return y cost times c 1 1 c 2 1 c 3 n + 1 c 4 n c 5 n c 6 1 T(n) = c 1 + c 2 + c 3 (n + 1) + c 4 n + c 5 n + c 6 Θ(n) c. loop invariant y = n (i+1) k=0 a k+i+1 x k Initialization i = n, y = 0 3 A[n + 1] =

20 20 2 Getting Started Maintenance y = a i+1 + a i+2 x + + a n x n (i+1) y = a i + a i+1 x + a i+2 x a n x n (i+1) 1 = Termination n [(i 1)+1] k=0 i = 1 n y = a k x k k=0 Problem 2-4 Inversions Inversions-Merge(A, p, q, r) a k+(i 1)+1 x k 1 n1 q p n2 r q 3 v 0 4 create arrays L[1.. n 1 ] and R[1.. n 2 ] 5 for i 1 to n 1 6 do L[i] A[p + i 1] 7 for j 1 to n 2 8 do R[j] A[q + j] 9 i 1 10 j 1 11 for k p to r 12 do if i > n 1 13 then A[k] R[j] 14 j j else if j > n 2 16 then A[k] L[i] 17 i i else if L[i] R[j] 19 then A[k] L[i] 20 i i v v + n 2 j else A[k] R[j] 23 j j return v

21 2.4 Problems 21 Inversions(A, p, r) 1 v 0 2 q p+r 2 3 if p < r 4 do v v + Inversions(A, p, q) 5 v v + Inversions(A, q + 1, r) 6 v v + Inversions-Merge(A, p, q, r) 7 return v Merge-Sort Θ(n lg n)

22

23 3 Growth of Functions 3.1 Asymptotic notation Exercise max(f(n), g(n)) = Θ(f(n) + g(n)) 0 c 1 f(n) + c 1 g(n) max(f(n), g(n)) c 2 f(n) + c 2 g(n) c 2 = 1 n 0 n n 0 max(f(n), g(n)) f(n) + g(n) c 1 = 1/2 and f(n) g(n) f(n) 1 2 f(n) 1 2 g(n) f(n) g(n) = 0 2 f(n) g(n) n 0 max(f(n), g(n)) 2 g(n) f(n) max(f(n), g(n)) = Θ(f(n) + g(n)) Exercise 3.1 5

24 24 3 Growth of Functions If f(n) = O(g(n)) and f(n) = Ω(g(n)) while n n 0 f(n) c 1 g(n) and while n n 0 f(n) c 2 g(n) while n max(n 0, n 0) f(n) = Θ(g(n)) 0 c 2 g(n) f(n) c 1 g(n), f(n) = Θ(g(n)) f(n) = O(g(n)) and f(n) = Ω(g(n)) Exercise Exercise o(g(n)) ω(g(n)) = f(n) = o(g(n)) c = 1 n > n 0 f(n) < g(n) f(n) = ω(g(n)) c = 1 n > n 0 f(n) > g(n) n > max(n 0, n 0) f(n) > g(n) f(n) < g(n) o(g(n)) ω(g(n)) = 3.2 Standard notations and common fuctions Exercise Prove equation (3.18). Stirling s approximation c n > n 0 n! ( n ) n ( 2πn 1 + c ) e n lg (n!) 1 2 lg 2π lg n + n lg n n lg e + lg (n + c) lg n (n > n 0)

25 3.2 Standard notations and common fuctions lg 2π + 1 lg n + n lg n n lg e + lg (n + c) lg n 2 = Θ(n lg n) + Θ(n) + Θ(lg n) + Θ(1) = Θ(n lg n) c n > n lg 2π lg n + n lg n n lg e + lg (n + c) lg n c n lg n n > max(n 0, n 0 ) lg (n!) c n lg n c n lg n lg (n!) lg n! = Θ(n lg n) Exercise i = 0 i = 1 F i = φi φ i 5 (i 3) Knuth Concrete Mathematics P F i+1 = F i + F i 1 = φi φ i 5 = + φi 1 5 = F i + φ i + φ i ( 2 φ i 1+ ) 5 2 φ ( i = φi φ i+1 φ i 1 1 ) 5 2

26 26 3 Growth of Functions Exercise i 0 F i+2 φ i i > 0 φ i < 1/2 5 1/2 < F i+2 φi+2 5 1/2 < F i+2 φ i φ2 5 ( 1/2 < F i+2 φ i ) φ i 1/2 < F i+2 φ i 0 < F i+2 φ i F i+2 > φ i i = 0 F 2 = φ 0 = 1 1 F i+2 φ i 1 :(

27 4 Recurrences 4.1 The substitution method Exercise T(n) = T( n/2 ) + 1 = O(lg n) T(n) = O(lg n) T(n) c lg n/2 + 1 = c lg n c + 1 c lg n (c 1) Exercise T(n) = 2T( n/2 ) + n = Ω(n lg n) T(n) 2c n lg n/2 + n 2 = cn lg n + (1 c)n cn lg n (0 < c 1) } T(n) = O(n lg n) T(n) = Θ(n lg n) T(n) = Ω(n lg n) Exercise (4.4) T(n) = O(n lg n + bn) (b > 0)

28 28 4 Recurrences T(n) c n 2 lg n/2 + cb 2 n + n = cn lg n cn + c b 2 n + n [ ( b = cn lg n + bn + c 2 1 ) ] + 1 b n cn lg n + bn (0 < c b 1 b b 2 1and 2 1 > 0) T(n) = O(n lg n + bn) n = 1 T(1) c lg 1 + b = b, b 1 O(n lg n + bn) = O(n lg n) Exercise T(n) = 2T( n/2 + 17) + n = O(n lg n) T(n) 2(n/2 + 17) lg (n/2 + 17) + n = cn lg n + cn lg (1 + 34/n) c(n + 34) + n cn lg n + cn lg (1 + 1/2) cn + n 34c (n 68) cn lg n + n(c lg 3 2c + 1) cn lg n Exercise (c 1 2 lg 3 ) T(n) = 2T( 2 n) + 1 n = 2 m T(2 m ) = 2T(2 m 2 )+1 S(m) = 2S(m/2)+ 1 S(m) = O(m b) S(m) cm 2b + 1 cm b (b 1) S(m) = O(m b) S(m) = Ω(m b), S(m) = Θ(m b) T(n) = T(2 m ) = S(m) = Θ(m b) = Θ(lg n b) = Θ(lg n) The recursion-tree method Exercise T(n) = 3T( n/2 ) + n

29 4.2 The recursion-tree method 29 n, 3 ( 2 ( ) k ) n, n,, n 2 }{{} lg n T(1), T(1),, T(1) }{{} n lg 3 T(n) = lg n 1 k=0 ( ) k 3 n + Θ(n lg 3 ) 2 ) lg n = n 1 ( Θ(n lg 3 ) 2 = 2n lg 3 2n + Θ(n lg 3 ) = O(n lg 3 ) T(n) = O(n lg 3 2/3n) ( n ) lg 3 T(n) 3c cn + n 2 cn lg 3 2/3n (c 5/3) Exercise Exercise lg n 1 cn, n, n } 2cn, 4cn, {{, cn 2 } lg n,, 1 lg n T(1), T(1),, T(1) }{{} n 2 T(n) = lg n 1 k=0 cn2 k + Θ(n 2 ) = cn 2 cn + Θ(n 2 ) = Θ(n 2 ) T(n) = Θ(n 2 n) = Θ(n 2 )

30 30 4 Recurrences ( n 2 T(n) 4d 4 n 2 ) + cn = d(n 2 n) + (c d)n d(n 2 n) = O(n 2 ) (d c) T(n) = Ω(n 2 ) T(n) = Θ(n 2 ) Exercise T(n) = T(n a) + T(a) + cn a T(a) T(n) = c[n + (n a) + (n 2a)+,, +a] + T(a)(n/a 1) }{{} n/a = c (a + n) n a + T(a) 2 a n T(a) = c 2a n2 + T(a) a n + c 2 T(a) = Θ(n 2 ) T(n) = Θ(n 2 ) T(a) > a 2 T(n) d(n 2 T(a)) = df(n) T(n) d(n a) 2 dt(a) + T(a) + cn = dn 2 T(a) + (c 2ad)n + (2 d)t(a) + da 2 dn 2 T(a) = O(n 2 ) ( ) c d max 2a, 2T(a) T (a) a ( 2 ) c d min 2a, 2T(a) T(n) dn 2 T(a), T(n) = T (a) a 2 Ω(n 2 ) T(n) = Θ(n 2 ) T(n) a 2 f(n) 1 Exercise α (1 α) 1/2 1 > α 1/2 1 T(a)

31 4.4 Problems 31 T(n) = cn log 1/α n + Θ(n log 1 α 2 ) 1 > α 1/2 n log 1 α 2 = ω(n) T(n) = Θ(n lg n) 1 > (1 α) 1/2 α = 1/2 Merge-Sort 4.3 The master method Exercise Exercise a 49 Exercise Master method T(n) = O(n 2 (lg n) 2 ) 4.4 Problems Problem 4-1 Recurrence examples a. T(n) = Θ(n 3 ) b. T(n) = Θ(n) c. T(n) = Θ(n 2 lg n) d. Master T(n) = Θ(n 2 lg n) e. T(n) = Θ(n log 2 7 ) f. T(n) = Θ( n) g T(n) = Θ(n 2 ) h. n = 2 m, T(n) = S(m) = S(m/2) + 1 = Θ(lg m) = Θ(lg lg n) Problem 4-2 Finding the missing integer a bits { T(a) if n = a T(n) = T(n) = 2T(n/2) + Θ(1) if n > a T(n) = 2T(n/2) + Θ(1), T(n) = O(n)

32 32 4 Recurrences Problem 4-3 Parameter-passing costs a. Time = Θ(1) T(n) = T(n/2) + Θ(1) T(N) = Θ(lg N) Time = Θ(N) Master T(N) = Θ(N lg N) Time = Θ(q p + 1) T(n) = T(n/2) + Θ(n) T(N) = N b. Time = Θ(1) T(n) = 2T(n/2) + n + Θ(1) T(N) = Θ(N lg N) Time = Θ(N) T(n) = 2T(n/2) + n + Θ(N) T(N) = Θ(N 2 ) + Θ(N lg N) + Θ(N) = Θ(N 2 ) Time = Θ(q p + 1) T(n) = 2T(n/2) + Θ(n) T(N) = Θ(N lg N) Problem 4-4 More recurrence examples a. T(n) = Θ(n log 2 3 ) b. T(n) = Θ(n) c. T(n) = Θ(n 2 n) d. T(n) = Θ(n lg n) e. T(n) = Θ(n) f. lg n ( ) k 1 7 T(n) = n + Θ(n lg 3 ) 8 k=1 ( ) k 1 7 < n + Θ(n lg 3 ) 8 k=1 = 8n + Θ(n lg 3 ) = O(n lg 3 )

33 4.4 Problems 33 g. 2 h. i. j. help me... T(n) = n = lg n + O(1) = O(lg n) T(n) = Θ(lg n!) T(n) = 2 lg[n(n 2)(n 4) (1)] + Θ(1) = Θ(lg n!) Problem 4-5 Fibonacci numbers a. F(z) = z + z 2 + 2z F i+2 z i+2 + zf(z) = z 2 + z 3 + 2z F i+1 z i+2 + z 2 F(z) = z 3 + 2z F i z i+2 + F i = F i 1 + F i 2 b. F(z) zf(z) z 2 F(z) = z F(z) = z + zf(z) + z 2 F(z) F(z) = z 1 z z 2 f(z) = 1 z z 2 1 z z 2 = 0 1± 5 2 ( 1 + ) ( ) f(z) = z + z 2 2 = (1 φz)(1 φz) F(z) = 1 ( ) φz 1 1 φz 2 A,Concrete Mathematics

34 34 4 Recurrences c. 1 1 α = 1 + α + α2 + α 3 + ( α < 1) z φz < 1 φz < 1 F(z) = i=0 1 5 (φ i φ i )z i d. φ i / 5 < 1 2 i φ i / 5 F i φ i / F i φ i / 5 e FT!! Problem 4-7 Monge arrays a. Monge A[i, j] + A[i + 1, j + 1] A[i, j + 1] + A[i + 1, j] A[i, j] + A[i + 1, j + l] A[i, j + l] + A[i + 1, j] (4.1) l = 1 A[i, j + l] + A[i + 1, j + l + 1] A[i, j + l + 1] + A[i + 1, j + l] (4.2) A[i + 1, j + l + 1] A[i + 1, j + l] A[i, j + l + 1] A[i, j + l] (4.3) (4.1) (4.3) A[i, j] + A[i + 1, j + (l + 1)] A[i, j + (l + 1)] + A[i + 1, j] (4.4) (4.1) A[i, j] + A[i + k, j + l] A[i, j + l] + A[i + k, j] (4.5) k = 1 (4.1) (4.1) A[i + k, j] + A[i + k + 1, j + l] A[i + k, j + l] + A[i + k + 1, j] (4.6)

35 4.4 Problems 35 A[i + k + 1, j + l] A[i + k, j + l] A[i + k + 1, j] A[i + k, j] (4.7) (4.5) (4.7) A[i, j] + A[i + (k + 1), j + l] A[i, j + l] + A[i + (k + 1), j] (4.8) (4.5) Monge array b c. f(i) > f(i + 1) Monge array A[i, f(i + 1)] + A[i + 1, f(i)] A[i, f(i)] + A[i + 1, f(i + 1)] (4.9) f(i) A[i + 1, f(i)] A[i + 1, f(i + 1)] A[i, f(i + 1)] A[i, f(i)] (4.9) Monge array f(i) f(i + 1) f(1) f(2) f(3) f(m) d. n O(n) 3 c 2i + 1 f(2i) f(2i + 2) O(f(2i + 2) f(2i) + 1) m n T(m, n) = f(2) + m 2 1 (f(2i + 2) f(2i) + 1) i=1 = f(m) + m 2 1 < m + n (f(m) n) = O(m + n) e. T(m, n) = T(m/2, n) + n + m 2 1 T(m, n) = n lg m + (m 2) (lg m 1) < n lg m + m = O(n lg m + m)

36

37 5 Probabilistic Analysis and Randomized Algorithms 5.1 The hiring problem Exercise n! which candidate is better than candidate best total order which candidate is better than the other total order 5.2 Indicator random variables Exercise P n n = n! C 1 1 Pn 1 n 1 Pr { } = Pn 1 n 1 P n n = 1 n n Pr { n } = 1 n! Exercise 5.2 2

38 38 5 Probabilistic Analysis and Randomized Algorithms 1, 2, 3, 4,, n n 1, 2, 3,, (n 1) n n 1 n n {x x 1, 2, 3,, (n 1)} 1, 1/2, 1/3, 1/(n 1) P = 1 n + 1 n n 1 n 1 = 1 ( n ) n 1 = 1 n ln(n 1) + 1 n O(1) Exercise X P X P p n [ n ] E[X SUM ] = E X P = = E[X P ] = ( ) 1 6 = 7 2 i=1 n E[X P ] i=1 n i=1 = 7n Exercise X i i X i = I { i } = { 1 i 0 i

39 5.2 Indicator random variables 39 X = X 1 + X 2 + X X n 1 1 n n 1 P1 n 1 P 2 n = 1 n n 2 P2 n 1 Pn 3 1 n [ n ] E[X] = E X i = = = 1 i=1 n E[X i ] i=1 n i=1 1 n ( 5.1) = 1 n 1 1 Exercise A n n! A Step 1. {1, 2,..., n} 1 X 1 = 0 Step X 2 = = 1 2 Step , 2 2, X 3 = Pr{ 1, 2 }( )+Pr{ 2, 1 }( ) = ( )2 1 A uniform random permutation 1, 2 2, 1 2 2

40 40 5 Probabilistic Analysis and Randomized Algorithms Step 4. i i 1 3 X i = 1 ((i 1) + (i 2) )(i 1)! i! = i 1 2 indicator random variable { i 1 X i = I{ i i 1 } 2 i 0 Pr{ i } = 1 Pr{ } = 0 < 1, 2,..., n > [ n 4 E [X i ] = E = i=1 X i ] n E[X i ] i=1 = n 1 2 n(n 1) = Randomized algorithms Exercise Professor Marceau indicator random variables indicator random variables... 5 Professor Marceau This one is a horrible pun. The question is about how to permute. Marcel Marceau is a famous mime. That is, he is mute. Sorry about that.

41 5.3 Randomized algorithms 41 Randomize-In-Place(A) 1 n length[a] 2 swap A[1] A[Random(1, n)] 3 for i 2 to n 4 do swap A[i] A[Random(i, n)] loop invariant A[1] A[1.. n] 1/n Exercise A 6 Exercise e...need help. Exercise B 7 A[i] offset offset 1 n 1/n A[i] B 1/n A B 6 Julius Kelp is the Nutty Professor, as played by Jerry Lewis in the original film of the same name. Professor Kelp has discovered a formula that changes his identity. The exercise is about how to permute in which the identity permutation cannot occur. This professor joke is the only one in the book in which the named professor really is a professor (though a fictional one). 7 This exercise is about cyclic permutations, and it of course refers to Lance Armstrong, winner of the Tour de France bicycle race.

42

43 Part II Sorting and Order Statistics

44

45 6 Heapsort 6.1 Heaps Exercise n = 2 h+1 1 n = 2 h Exercise n 2 h n 2 h+1 1 h lg n lg 2 h+1 1 < h + 1 h lg n < h + 1 h = lg n Exercise n 2 n Left ( n ) = 2 ( n ) > n n n

46 46 6 Heapsort 6.2 Maintaining the heap property Exercise Max-Heapify(A, i) 1 index i 2 while index heap-size[a]/2 3 do l Left(index) 4 r Right(index) 5 if l heap-size[a] and A[l] > A[index] 6 then largest l 7 else largest index 8 if r heap-size[a] and A[r] > A[largest] 9 then largest r 10 if largest index 11 then exchange A[index] A[largest] 12 index largest 13 else break Exercise n Max-Heapify lg n Ω(h) = Ω(lg n) 6.3 Building a heap Exercise Max-Heapify max-heap 1 Exercise help me 6.4 The heapsort algorithm Exercise A Buld-Max-Heap O(n) n 1 Max-Heapify Heap-Sort O(n lg n) O(n lg n)

47 6.5 Priority queues 47 Exercise Max-Heapify Θ(lg n) Heap-Sort nθ(lg n) + O(n) = Θ(n lg n) Ω(n lg n) 6.5 Priority queues Exercise Heap-Minimum(A) 1 return A[1] Heap-Extract-Min(A) 1 if heap-size[a] < 1 2 then error heap underflow 3 min A[1] 4 A[1] A[heap-size[A]] 5 A[heap-size] A[heap-size[A]] 1 6 Min-heapify(A, 1) 7 return min Heap-Decrease-Key(A, i, key) 1 if A[i] < key 2 then error new key is greater than current key 3 A[i] key 4 while i > 1 and A[i] < Parent(i) 5 do exchange A[i] A[Parent(i)] 6 i Parent(i) Min-Heap-Insert(A, key) 1 heap-size[a] heap-size[a] A[heap-size[A]] 3 Heap-Decrease-Key(A, heap-size[a], key) Exercise A[heap-size[A]] key Heap-Increase-Key Exercise min-priority queue queue Min-Heap-Insert Heap-Minimum max-priority queue

48 48 6 Heapsort Exercise Heap-Delete(A, i) 1 if i > heap-size[a] 2 then error heap underflow 3 A[i] A[heap-size[A]] 4 heap-size heap-size 1 5 if A[i] > A[Parent(i)] 6 then Heap-Increase-Key(A, i, A[i]) 7 else Max-Heapify(A, i) Max-Heapify Heap-Increase-Key O(lg n) Exercise Merge-Sort k min-heap root Min-Master-Heapify n minheap root min-heap Min-Sub-Heapify L[1.. k] k L[1.. k][n ] n Master-Heap-Extract-Min(L) 1 min L[1][1] 2 root 3 L[1][1] 4 Min-Sub-Heapify(L[1], 1) 5 Min-Master-Heapify(L, 1) 6 return min Min-Sub-Heapify Min-Master-Heapify 1 O(lg k + lg n k ) n k k -Way-Merge(A, L) 1 Build-Min-Heap(L) 2 for i 1 to n 3 A[i] Master-Heap-Extract-Min(L) 1 C L

49 6.6 Problems 49 T(n) = k+n lg k+n 1 lg n 1 + n 2 lg n n k lg n }{{ k n lg k } k n 1 k = n/n 1 T(n) = n/n 1 + n lg n/n 1 + n lg n 1 = O(n lg n) = O(n lg n/n 1 ) = O(n lg k) n 1 n O(n lg k) 6.6 Problems Problem 6-1 Building a heap using insertion a. Exercise b. Max-Heap-Insert O(lg n) n O(n lg n) Problem 6-2 Analysis of d-ary heaps a. b. log d n c. Max-Heapify O(1) O(log d n) 2 d. O(log d n) e. O(log d n) Problem 6-3 Young tableau a b. A[1, 1] A[1, 1] A[1, 1] = A[m, n] A[m, n] < c. Young tableaus A[i, j] (n j) (m i+1) 1 (m i) (i, j) 2 O(d log d n)? d

50 50 6 Heapsort Right(i, j) 1 return i, j + 1 Down(i, j) 1 return i + 1, j Up(i, j) 1 return i 1, j Left(i, j) 1 return i, j 1 A[i, j] Young-Tableaufy(A, p) 1 r Right(p) 2 b Down(p) 3 if b y m and A[p] > A[b] 4 then smallest b 5 else smallest p 6 if r x n and A[smallest] > A[r] 7 then smallest r 8 if smallest p 9 then exchange A[p] A[smallest] 10 Young-Tableaufy(A, smallest) Tableau-Extract-Min(A) 1 if A[1, 1] = 2 then error array is empty 3 min A[1, 1] 4 A[1, 1] 5 Young-Tableaufy(A, (1, 1)) 6 return min T(p) = T(p 1) + Θ(1) if p > 0 T(p) = O(m + n) [1, 1] [m, n] O(m + n)

51 6.6 Problems 51 d. Tableau-Decrease-key(A, p, key) 1 if key A[p] 2 then error new key is greater than current key 3 A[p] key 4 while A[p] < A[Left(p)] or A[p] < A[Up(p)] 5 do if p x < 1 or p y < 1 6 then if A[p] < A[Left(p)] 7 then exchange A[p] A[Left(p)] 8 p Left(p) 9 if A[p] < A[Up(p)] 10 then exchange A[p] A[Up(p)] 11 p Up(p) 12 else break A[m, n] key Tableau- Decrease-key Tableau-Insert-Key(A, key) 1 if A[m, n] < 2 then error Array is full 3 Tableau-Decrease-key(A, (m, n), key) e. n 2 young-tableau Young-Tableau-Sort(A) 1 creat n n array Y 2 for i 1 to n 2 3 do Y[i] A[i] 4 Build-Tableau(Y) 5 for j 1 to n 2 6 do A[i] Tableau-Extract-Min(Y) Build-Tableau n 2 Tableau-Extract- Min Build-Tableau O(n 3 ) 3 Tableau-Extract-Min O(n) n 2 O(n 3 ) f. 3

52 52 6 Heapsort 6.7 The C implementations of algorithms gcc (GCC) 3.2.3, glibc 2.3.2, GNU gdb 5.3, linux kernel #define MAXSIZE 10 2 #define PARENT(i) ( (i) / 2) 3 #define LEFT(i) ( (i) << 1) 4 #define RIGHT(i) ( (i << 1) + 1) 5 typedef int INDEX; 6 typedef int ELEMTYPE; 8 void max_heapify (ELEMTYPE a[], INDEX i, int heap_size); 9 void build_max_heap (ELEMTYPE a[]); 10 void heap_sort (ELEMTYPE a[]); 12 void 13 max_heapify (ELEMTYPE a[], INDEX i, int heap_size) 14 { 15 INDEX l, r, largest; 16 ELEMTYPE temp; 17 l = LEFT(i); 18 r = RIGHT(i); 20 if (l <= heap_size && *(a + i 1) < *(a + l 1)) 21 largest = l; 22 else 23 largest = i; 25 if (r <= heap_size && *(a + largest 1) < *(a + r 1)) 26 largest = r; 28 if (largest!= i) 29 { 30 temp = *(a + i 1); 31 *(a + i 1) = *(a + largest 1); 32 *(a + largest 1) = temp; 33 max_heapify (a, largest, heap_size); 34 } 35 } 37 void 38 build_max_heap (ELEMTYPE a[]) 39 { 40 INDEX i; 42 for (i = MAXSIZE/2; i > 0; i ) 43 max_heapify (a, i, MAXSIZE); 44 }

53 6.7 The C implementations of algorithms void 47 heap_sort (ELEMTYPE a[]) 48 { 49 int i; 50 int heap_size; 51 ELEMTYPE temp; 53 heap_size = MAXSIZE; 54 build_max_heap (a); 55 for (i = MAXSIZE ; i > 0; i ) 56 { 57 temp = a[heap_size 1]; 58 a[heap_size 1] = a[0]; 59 a[0] = temp; 60 heap_size = 1; 61 max_heapify (a, 1, heap_size); 62 } 63 }

54

55 7 Quicksort 7.1 Description of quicksort Exercise Partition(A, p, r) 1 x A[r] 2 i p 1 3 counter 0 4 for j p to r 1 5 do if A[j] < x 6 then i i exchange A[i] A[j] 8 if A[j] = x 9 then counter counter exchange A[i + 1] A[r] 11 if counter = p r 12 then return (p + r)/2 13 else return i Exercise p r = n Θ(1) Θ(n) 7.2 Performance of quicksort Exercise T(n) = Θ(n 2 ) Θ(n) = c n c

56 56 7 Quicksort T(n) c(n 1) 2 + c n c = cn 2 + (c 2c)n cn 2 (c c /2) Exercise Θ(n 2 ) 7.3 A randomized version of quicksort Exercise Random pivot n n Random Θ(n) 7.4 Analysis of quicksort Exercise T(n) c max 0 q n 1 (T(q) + T(n q 1)) + Θ(n) = cn 2 c(2n 1) + Θ(n) = cn 2 c(2n 1) + b(2n 1) cn 2 (0 < c b) T(n) = Ω(n 2 ) q 2 + (n q 1) 2 q = 0 q = n Problems 7-1 Hoare partition correctness a. so easy.

57 7.5 Problems 57 b. A[j.. r] x A[p.. i] x x = A[p] x 9 i j i > j i j i = j + 1 i = j i = j + 1 j i r p (r > p) i = j 5, 7 j r 1 < r i q + 1 > q i = j p + 1 i = j r 1 i = j + 1 p j < r p < i r i, j A[p.. r] c. p j < r d. e. Hoare-Quicksort(A, p, r) 1 if p < r 2 then q Hoare-Partition(A, p, r) 3 Hoare-Quicksort(A, p, q) 4 Hoare-Quicksort(A, q + 1, r) Problem 7-4 Stack depth for quick sort a. p q + 1 Quicksort(A, q + 1, r) b. T(n) = T(n 1) + T(0) + Θ(n) Θ(n) Θ(lg n) c. Sedgewick Algorithms O(lg n) n S(n) in C Parts 1 4 n/2 P314 1 S(n) S(n/2) Master Method S(n) = O(lg n)

58 58 7 Quicksort 7.6 The C implementations of algorithms gcc (GCC) 3.2.3, glibc 2.3.2, GNU gdb 5.3, linux kernel #include <stdio.h> 3 #define MAXSIZE 20 5 typedef int ELEMTYPE; 6 typedef int INDEX; 8 INDEX partition (ELEMTYPE a[], INDEX p, INDEX r); 9 void quicksort (ELEMTYPE a[], INDEX p, INDEX r); 11 INDEX 12 partition (ELEMTYPE a[], INDEX p, INDEX r) 13 { 14 ELEMTYPE x, temp; 15 INDEX i, j; 17 x = a[r]; 18 i = p 1; 20 for (j = p; j <= r 1; j++) 21 { 22 if (a[j] <= x){ 23 i += 1; 24 temp = a[i]; 25 a[i] = a[j]; 26 a[j] = temp; 27 } 28 } 30 temp = a[i + 1]; 31 a[i + 1] = a[r]; 32 a[r] = temp; 34 return (i + 1); 35 } 37 void 38 quicksort (ELEMTYPE a[], INDEX p, INDEX r) 39 { 40 INDEX q; 41 while (p < r){ 42 q = partition (a, p, r); 43 quicksort (a, p, q 1); 44 p = q + 1;

59 45 } 46 } 7.6 The C implementations of algorithms 59

60

61 8 Sorting in Linear Time 8.1 Lower bounds for sorting Exercise Θ(n) Exercise n n lg k lg n k=1 k=1 = n lg n = O(n lg n) (8.1) n/2 n n lg k = lg k + lg k k=1 k=1 n lg k k=n/2+1 k=n/2+1 (8.2) n 2 lg n 2 = Ω(n lg n) Θ(n lg n)

62 62 8 Sorting in Linear Time Exercise n! h lg n! 2 = Ω(n lg n) Ω(n lg n) n!/n h lg n! n = lg (n 1)! = Ω(n lg n) 1 2 n n lg n! 2 n = lg n! n = Ω(n lg n) Θ(n) = Ω(n lg n) 8.2 Counting sort Exercise A[i] = A[j] (i < j) 9 A[j] A[i] B 11 A[i] A[j] Exercise 8.2 3

63 8.3 Radix sort 63 Exercise Counting(A, a, b) 1 if a < b 2 then error a must be less than or equal to b 3 for i 0 to k 4 C[i] 0 5 for j 1 to n 6 C[A[j]] C[A[j]] for i 1 to k 8 C[i] C[i] + C[i 1] 9 if a = 0 10 then return A[b] 11 else return A[b] A[a 1] Counting-Sort Θ(n) 8.3 Radix sort Exercise n 2 1 n 0 n 1 (n 2 1) mod n 0 n 1 Radix-Sort Counting-Sort O(n + n) = O(n) Counting-Sort O(n + n) O(n)

64 64 8 Sorting in Linear Time 8.4 The C implementations of algorithms gcc (GCC) 3.2.3, glibc 2.3.2, GNU gdb 5.3, linux kernel #include <stdio.h> 2 #include <stdlib.h> 3 #include <ctype.h> 4 #include <sys/time.h> 6 #define MAXSIZE 20 8 typedef int ElemType; 9 ElemType grandom (ElemType p, ElemType r); 10 void count_sort (ElemType a[], ElemType b[], 11 ElemType lower, ElemType upper); 12 int isdigitstr (char *s); 14 /* R e t u r n r a n d o m n u m b e r f r o m p to r */ 15 ElemType 16 grandom (ElemType p, ElemType r) 17 { 18 ElemType v; 19 /* G e n e r a t e r a n d o m n u m b e r f r o m p to r */ 20 v = p + rand () % (r p + 1); 21 return (v); 22 } 24 void 25 count_sort (ElemType a[], ElemType b[], 26 ElemType lower, ElemType upper) 27 { 28 int i, j, range; 29 ElemType *pt; 31 range = upper lower + 1; 32 pt = (ElemType *) malloc (range * sizeof (ElemType)); 33 for (i = 0; i < range; ++i) 34 pt[i] = 0; 36 for (j = 0; j < MAXSIZE; ++j) 37 pt[a[j] lower] = pt[a[j] lower] + 1; 39 for (i = 1; i < range; ++i) 40 pt[i] = pt[i] + pt[i 1]; 42 for (j = MAXSIZE 1; j >= 0; j) { 43 b[pt[a[j] lower] 1] = a[j]; 44 pt[a[j] lower] = 1;

65 8.4 The C implementations of algorithms } 46 free (pt); 47 } 49 int 50 isdigitstr (char *s) 51 { 52 int v = 0; 53 while (*s!= \0 ) { 54 if (isdigit (*s)) 55 v = 1; 56 else { 57 v = 0; 58 break; 59 } 60 s++; 61 } 62 return (v); 63 } 65 int 66 main (int argc, char *argv[]) 67 { 68 int i; 69 unsigned int seed; 70 struct timeval tv; 71 ElemType a[maxsize], b[maxsize], lower, upper; 73 if (argc!= 3) { 74 printf ("Usage:count_sort n1 n2\n"); 75 printf ("n1 and n2 should be nonegative integer, and n1 < n2\n"); 76 exit (1); 77 } 78 else { 79 while ( argc > 0 && (*++argv)[0] == ) { 80 if (argc == 2 && isdigitstr (*argv + 1)) 81 lower = atoi (*argv + 1); 82 else if (argc == 1 && isdigitstr (*argv + 1)) 83 upper = atoi (*argv + 1); 84 else { 85 printf ("Usage:count_sort n1 n2\n"); 86 printf ("n1 and n2 should be nonegative integer, and n1 < n2\n"); 87 exit (1); 88 } 89 } 90 }

66 66 8 Sorting in Linear Time 92 if (argc!= 0) { 93 printf ("Usage:count_sort n1 n2\n"); 94 printf ("n1 and n2 should be nonegative integer, and n1 < n2\n"); 95 exit (1); 96 } 98 if (lower > upper) { 99 printf ("The lower must be smaller than the upper\n"); 100 exit (1); 101 } 104 /* M a k e s e e d */ 105 gettimeofday (&tv, NULL); 106 seed = tv.tv_sec + tv.tv_usec; 107 srand (seed); 109 for (i = 0; i < MAXSIZE; ++i) { 110 a[i] = grandom (lower, upper); 111 b[i] = 0; 112 } 114 for (i = 0; i < MAXSIZE; ++i) 115 printf ("%d ", a[i]); 116 printf ("\n"); 118 count_sort (a, b, lower, upper); 120 for (i = 0; i < MAXSIZE; ++i) 121 printf ("%d ", b[i]); 122 printf ("\n"); 123 return (0); 124 }

67 9 Medians and Order Statistics 9.1 Minimum and maximum 9.2 Selection in expected linear time Exercise q = p q = r 0 i = 1 i = p q i < 1 i > r 0 Exercise Randomized-Select(A, p, r, i) 1 if p = r 2 then return A[p] 3 L p 4 R r 5 q Randomized-Partition(A, L, R) 6 k q L while i k 8 do if i < k 9 then R q 1 10 q Randomized-Partition(A, L, R) 11 elseif i > k 12 then L q i i k 14 q Randomized-Partition(A, L, R) 15 k q L return A[q]

68 68 9 Medians and Order Statistics Exercise pivot 9.3 Selection in worst-case linear time Exercise ( 1 n 4 2) 2n T(n) T(n) cn { Θ(1) if n n 0 T( n/7 ) + T(5n/7 + 8) + O(n) if n > n 0 (9.1) T(n) cn/7 + c + 5cn/7 + 8c + an = 6cn/7 + 9c + an = cn + ( cn/7 + 9c + an) c 7a(n/(n 63)) n > n 0 = 126 n n 0 c 14a 3 T(n) T( n/3 ) + T(2n/3 + 4) + O(n) O(n lg n) P70 Exercise Select pivot Quicksort O(n lg n) Exercise black box T(n) = T(n/2) + O(n) Master Method T(n) = O(n)

69 9.3 Selection in worst-case linear time 69 Exercise Select k/2 k/2 k T(n) = 2T(n/2) + O(n) (n k) T(n) = O(n lg k) B[1.. k 1] List-Quantiles(A, B, p, r, k) 1 if (r p + 1 (k 1)) mod k 0 2 then error Array can t be divided into k equal-size 3 block-size r p+1 (k 1) k 4 q (block-size +1) k/2 5 base 6 base p 1 block-size +1 7 if p r > block-size 8 then B[base + k/2 ] Select(A, p, r, q) 9 m Partition(A, p, r, B[base + k/2 ]) 10 List-Quantiles(A, B, p, m 1, k/2 ) 11 List-Quantiles(A, B, m + 1, r, k k/2 ) Exercise n 2n n n 1 n X[ (n+1)/2 ] Y[ (n+1)/2 ] X[ (n + 1)/2 ] < Y[ (n + 1)/2 ] X[ (n + 1)/2.. n] Y[1.. (n + 1)/2 ] 2 4 Θ(1) n 1 n Find-Median(X, Y, p x, r x, p y, r y ) 1 q (p + r)/2 2 if r x p x + 1 = r y p y + 1 = 2 3 then Z[1] X[p x ] 4 Z[2] X[r x ] 5 Z[3] Y[p y ] 6 Z[4] Y[r y ] 7 Insertion-Sort(Z) 8 return Z[2] 9 if X[q] < Y[q] 10 then return Find-Median(X, Y, q, r x, p y, q) 11 else return Find-Median(X, Y, p x, q, q, r y )

70 70 9 Medians and Order Statistics T(n) = T(n/2) + Θ(1) Master Method T(n) = Θ(lg n) Exercise n L 0 x L = L 0 + (n + 1)x nx > L 0 O(n) O(n) Problems Problem 9.1 Largest i numbers in sorted order a. n O(n lg n) i O(i) T(n) = O(n lg n + i) b. O(n) Extract-Max O(i lg n) T(n) = O(n + i lg n) c. i O(n) Partition O(n) i i lg i T(n) = O(n + i lg i) i n i n 1 Professor Olay The exercise asks about an oil pipeline, and the professor s name refers to the cosmetic product Oil of Olay.

71 Part III Data Structures

72

73 10 Elementary Data Structures 10.1 Stacks and queues Exercise top[s 1 ] = 0 top[s 2 ] = n top[s 1 ] = top[s 2 ] Exercise Enqueue(Q, x) 1 if tail[q] = length[q] and head[q] = 1 2 then error Queue is overflow 3 elseif tail[q] + 1 = head[q] 4 then error Queue is overflow 5 Q[tail[Q]] x 6 if tail[q] = length[q] 7 then tail[q] 1 8 else tail[q] tail[q] + 1 Dequeue(Q) 1 if head[q] = tail[q] 2 then error Queue is underflow 3 x Q[head[Q]] 4 if head[q] = length[q] 5 then head[q] 1 6 else head[q] head[q] return x

74 74 10 Elementary Data Structures Exercise Q 1, Q 2 Push Q 1 Enqueue Pop Q 2 Q 1 Enqueue(Q 1, Dequeue(Q 1 )) Q 1 Enqueue(Q 2, Dequeue(Q 1 )) Q 1 Q 2 Dequeue(Q 2 ) Pop O(n) 10.2 Linked lists Exercise Push(L, x) 1 next[x] head[l] 2 head[l] x Pop(L) 1 if head[l] = nil 2 then error Stack is underflow 3 x head[l] 4 head[l] next[head[l]] 5 return x Exercise tail[l] = nil[l] Enqueue(L, x) 1 next[tail[l]] x 2 next[x] nil Dequeue(L) 1 if tail[l] = nil[l] 2 then error Queue is underflow 3 x next[nil[l]] 4 next[nil[l]] next[x] 5 if next[nil[l]] = nil 6 then tail[l] nil[l] 7 return x

75 10.2 Linked lists 75 Exercise Insert(L, x) 1 y head[l] 2 while next[y] head[l] 3 do y next[y] 4 next[y] x 5 next[x] head[l] 6 head[l] x Delete(L, x) 1 y head[l] 2 while next[y] x 3 do y next[y] 4 next[y] next[x] 5 if x = head[l] 6 then head[l] next[head[l]] Search(L, k) 1 x head[l] 2 while key[x] k and next[x] = head[l] 3 do x next[x] 4 if key[x] = k 5 then return x 6 else return nil O(n) Exercise Union(S 1, S 2 ) 1 next[prev[head[s 1 ]]] S 2 2 next[prev[head[s 2 ]]] S 1 3 x prev[head[s 1 ]] 4 prev[head[s 1 ]] prev[head[s 2 ]] 5 prev[head[s 2 ]] x 6 return S 1 Exercise

76 76 10 Elementary Data Structures Reverse-List(L) 1 x head[l] 2 p nil 3 while x nil 4 do y next[x] 5 next[x] p 6 p x 7 x y 8 return p 10.3 Implementing pointers and objects Exercise Allocate-Object() 1 if free = nil 2 then error out of space 3 else x free 4 free A[x + 1] 5 returnx Free-Object(x) 1 A[x + 1] free 2 free x 10.4 Representing rooted trees Exercise Traversal-Binary-Tree(x) 1 if x nil 2 then print key[x] 3 Traversal-Binary-Tree(left[x]) 4 Traversal-Binary-Tree(right[x]) T(n) = 2T(n/2) + Θ(1) Master Method O(n) Exercise S S

77 Preorder-Traversal-Binary-Tree(x) 1 if x nil 2 then Push(S, x) 3 while Stack-Empty(S) = false 4 do y Pop(S) 5 print key[y] 6 if right[y] nil 7 then Push(S, right[y]) 8 if left[y] nil 9 then Push(S, left[y]) 10.4 Representing rooted trees 77 S t S e Step.1 S t Step.2 S e Step.3 S t Push(S t, t) S e S t Inorder-Traversal-Binary-Tree(x) 1 if x nil 2 then Push(S t, right[x]) 3 Push(S e, x) 4 Push(S t, left[x]) 5 while Stack-Empty(S e ) = false 6 do t Pop(S t ) 7 if t = nil 8 then e Pop(S e ) 9 print e 10 t Pop(S t ) 11 if t = nil 12 then Push(S t, t) 13 else Push(S t, right[t]) 14 Push(S e, t) 15 Push(S t, left[t]) S e

78 78 10 Elementary Data Structures Postorder-Traversal-Binary-Tree(x) 1 if x nil 2 then Push(S e, x) 3 Push(S t, right[x]) 4 Push(S t, left[x]) 5 while Stack-Empty(S e ) = false 6 do l Pop(S t ) 7 if l = nil 8 then r Pop(S t ) 9 if r = nil 10 then e Pop(S e ) 11 print e 12 if l = nil and r = nil 13 then Push(S t, nil) 14 else if l = nil 15 then t r 16 Push(S t, l) 17 else t l 18 Push(S t, right[t]) 19 Push(S t, left[t]) 20 Push(S e, t) Exercise Print-Tree(x) 1 if x nil 2 then print key[x] 3 Print-Tree(left-child[x]) 4 Print-Tree(right-sibling[x]) 2 O(1) n O(n)

79 10.5 Problems Problems Problem 10-1 Comparisons among lists unsorted, singly linked sorted, singly linked unsorted, doublly linked sorted, doublly linked Search(L, k) O(n) O(n) O(n) O(n) Insert(L, x) O(1) O(n) O(1) O(n) Delete(L, x) O(n) O(n) O(1) O(1) Successor(L, x) O(n 2 ) O(1) O(n 2 ) O(1) Predecessor(L, x) O(n 2 ) O(n) O(n 2 ) O(1) Minimum(L) O(n) O(1) O(n) O(1) Maximum(L) O(n) O(n) O(n) O(1)

80 80 10 Elementary Data Structures 10.6 The C implementations of algorithms gcc (GCC) 3.2.3, glibc 2.3.2, GNU gdb 5.3, linux kernel #include <stdio.h> 2 #include <stdlib.h> 4 /* T h e e l e m e n t t y p e of t r e e n o d e s k e y f i e l d. */ 5 typedef char ElemType; 6 typedef struct tree_node *Treeptr; 7 typedef struct stack *Stackptr; 8 typedef struct stack 9 { 10 Treeptr *data; 11 int top; 12 } Stack; 14 typedef struct tree_node 15 { 16 ElemType key; 17 Treeptr left; 18 Treeptr right; 19 } TreeNode; 21 Treeptr build_tree (); 22 void preorder_traversal (Treeptr x); 23 void inorder_traversal (Treeptr x); 24 void postorder_traversal (Treeptr x); 25 void preorder_norecursion (Treeptr x); 26 void inorder_norecursion (Treeptr x); 27 void postorder_norecursion (Treeptr x); 28 void visit (Treeptr node); 30 Stackptr build_stack (int max); 31 int stack_empty (Stackptr s); 32 void push (Stackptr s, Treeptr x); 33 Treeptr pop (Stackptr s); 35 void 36 preorder_norecursion (Treeptr x) 37 { 38 Stackptr s1; 39 Treeptr t; 40 if (x!= NULL) { 41 s1 = build_stack (20); 43 push (s1, x); 44 while (!stack_empty (s1)) {

81 45 t = pop (s1); 46 visit (t); 47 if (t >right!= NULL) 48 push (s1, t >right); 49 if (t >left!= NULL) 50 push (s1, t >left); 51 } 52 } 53 } 55 void 56 postorder_norecursion (Treeptr x) 57 { 58 Stackptr s1, s2; 59 Treeptr l, r, t, e; 60 if (x!= NULL) { 61 s1 = build_stack (20); 62 s2 = build_stack (20); 64 push (s2, x); 65 push (s1, x >right); 66 push (s1, x >left); 67 while (!stack_empty (s2)) { 68 if ((l = pop (s1)) == NULL) 69 if ((r = pop (s1)) == NULL) { 70 e = pop (s2); 71 visit (e); 72 } 74 if (l == NULL && r == NULL) 75 push (s1, NULL); 76 else { 77 if (l == NULL) { 78 t = r; 79 push (s1, l); 80 } 81 else 82 t = l; 83 push (s1, t >right); 84 push (s1, t >left); 85 push (s2, t); 86 } 87 } 88 } 89 } 91 void 92 inorder_norecursion (Treeptr x) 93 { 10.6 The C implementations of algorithms 81

82 82 10 Elementary Data Structures 94 Stackptr s1, s2; 95 Treeptr t, e; 97 if (x!= NULL) { 98 s1 = build_stack (20); 99 s2 = build_stack (20); 100 push (s2, x); 101 push (s1, x >right); 102 push (s1, x >left); 103 while (!stack_empty (s2)) { 104 t = pop (s1); 105 if (t == NULL) { 106 e = pop (s2); 107 visit (e); 108 t = pop (s1); 109 } 111 if (t == NULL) 112 push (s1, t); 113 else { 114 push (s1, t >right); 115 push (s2, t); 116 push (s1, t >left); 117 } 118 } 119 } 120 } 122 Stackptr 123 build_stack (int max) 124 { 125 Stackptr s; 126 s = malloc (sizeof (struct stack)); 127 s >data = malloc (max * sizeof (Treeptr)); 128 s >top = 1; 129 return s; 130 } 132 int 133 stack_empty (Stackptr s) 134 { 135 if (s >top == 1) 136 return (1); 137 else 138 return (0); 139 } 141 void 142 push (Stackptr s, Treeptr x)

83 10.6 The C implementations of algorithms { 144 s >top += 1; 145 s >data[s >top] = x; 146 } 148 Treeptr 149 pop (Stackptr s) 150 { 151 if (stack_empty (s)) { 152 printf ("stack is underflow\n"); 153 return NULL; 154 } 155 else { 156 s >top = 1; 157 return s >data[s >top + 1]; 158 } 159 } 161 /* B u i l d a b i n a r y t r e e in p r e o r d e r t r a v e r s a l. */ 162 Treeptr 163 build_tree () 164 { 165 Treeptr tp; 166 ElemType c; 168 scanf ("%c", &c); 169 /* U s e " \ " to i n d i c a t e N U L L t r e e. */ 170 if (c == \\ ) { 171 tp = NULL; 172 return tp; 173 } 174 else if (tp = (TreeNode *) malloc (sizeof (TreeNode))) { 175 tp >key = c; 176 tp >left = build_tree (); 177 tp >right = build_tree (); 178 return tp; 179 } 180 else { 181 printf ("e...rpwt\n"); 182 return NULL; 183 } 184 } 186 void 187 visit (Treeptr node) 188 { 189 printf ("%c ", node >key); 190 }

84 84 10 Elementary Data Structures 192 void 193 preorder_traversal (Treeptr x) 194 { 195 if (x!= NULL) { 196 visit (x); 197 preorder_traversal (x >left); 198 preorder_traversal (x >right); 199 } 200 } 202 void 203 inorder_traversal (Treeptr x) 204 { 205 if (x!= NULL) { 206 inorder_traversal (x >left); 207 visit (x); 208 inorder_traversal (x >right); 209 } 210 } 212 void 213 postorder_traversal (Treeptr x) 214 { 215 if (x!= NULL) { 216 postorder_traversal (x >left); 217 postorder_traversal (x >right); 218 visit (x); 219 } 220 } 222 int 223 main () 224 { 225 Treeptr p; 226 p = build_tree (); 228 preorder_traversal (p); 229 printf ("\n"); 230 inorder_traversal (p); 231 printf ("\n"); 232 postorder_traversal (p); 233 printf ("\n"); 234 preorder_norecursion (p); 235 printf ("\n"); 236 inorder_norecursion (p); 237 printf ("\n"); 238 postorder_norecursion (p); 239 printf ("\n"); 240 return (0);

85 241 } 10.6 The C implementations of algorithms 85

86

87 11 Hash Tables 11.1 Direct-address tables Exercise Direct-Address-Search(T, k) 1 if T[k] 0 2 then return k 3 else return nil Direct-Address-Insert(T, x) 1 T[x] 1 Direct-Address-Delete(T, x) 1 T[x] Hash tables Exercise indicator random variable X kl = I{k l and h(k) = h(l)} P{k l and h(k) = h(l)} = 1/m [ n ] n n(n 1) E X kl = 2m k=1 l=k+1 = α(n 1) 2

88 88 11 Hash Tables 11.3 Hash fuctions Exercise m = mod 12 = mod 12 = 8 2. (8 8) mod 12 = 4 3. (4 8) mod 12 = 8 4. (8 8) mod 12 = 4 5. (4 (99 mod 12)) mod 12 = mod 12 = 4 (0 + 4) mod 12 = 4 Exercise m = 2 p 1 k = a n (m + 1) n + a n 1 (m + 1) n a 1 (m + 1) + a 0 k mod m = (a n + a n a 1 + a 0 ) mod m x y Exercise MIT-Scheme 1 (define h 2 (lambda (k) 3 (floor (* 1000 ( (* k (/ ( (sqrt 5) 1) 2)) (floor (* k (/ ( (sqrt 5) 1) 2)))))))) 4 (list (h 61) (h 62) (h 63) (h 64) (h 65)) 1 ; V a l u e 5 : ( )

89 11.4 Open addressing Open addressing Exercise Exercise Hash-Delete(T, k) 1 i 0 2 repeat j h(k, i) 3 if T[j] = k 4 then T[j] deleted 5 i i until T[j] = nil or j = m Hash-Insert(T, k) 1 i 0 2 repeat j h(k, i) 3 if T[j] = nil or T[j] = deleted 4 then T[j] k 5 return j 6 i i until i = m 8 error hash table overflow

90

91 12 Binary Search Trees 12.1 What is a binary search tree? Exercise min-heap O(n) Heap-Sort Θ(n lg n) Exercise Inorder-Tree-Walk(x) 1 find the minimum element. 2 while left[x] nil 3 do x left[x] 4 print x 5 find successor successively until traversal the entire tree. 6 repeat if right[x] nil 7 then x right[x] 8 while left[x] nil 9 do x left[x] 10 print x 11 else y p[x] 12 while y nil and x = right[y] 13 do x y 14 y p[y] 15 if y nil 16 then print y 17 x y 18 until y = nil

92 92 12 Binary Search Trees Exercise binary search tree Ω(n lg n) 12.2 Querying a binary search tree Exercise Tree-Minimun(x) 1 if left[x] = nil 2 then return x 3 else return Tree-Minimum(left[x]) Exercise Tree-Predecessor(x) 1 if left[x] nil 2 then return Tree-Maximum(left[x]) 3 y p[x] 4 while y nil and x = left[y] 5 do x y 6 y p[y] 7 return y Exercise easy 12.3 Insertion and deletion Exercise Tree-Insert(x, y, z) 1 if x = nil 2 then p[z] y 3 if y = nil 4 then x z tree is empty. 5 else if key[y] < key[z] 6 then right[y] z 7 else left[y] z 8 else y x 9 if key[x] < key[z] 10 then Tree-Insert(right[x], y, z) 11 else Tree-Insert(left[x], y, z)

93 12.4 Problems 93 Exercise Exercise Θ(n) Tree-Insert O(h) Θ(n) T = Θ( n) + Θ(n) = Θ(n 2 ) + Θ(n) = Θ(n 2 ) Θ(lg n) T = Θ(lg 1 + lg2 + lg3 + + lg n) + Θ(n) = Θ(lg n!) + Θ(n) = Θ(n lg n) + Θ(n) = Θ(n lg n) 12.4 Problems Problem 12 1 a. O(n) O(n) b. O(n/2) c. O(1) d. O(n) O(n lg n) Problem 12 2

94

95 13 Red-Black Trees 13.1 Properties of red-black trees Exercise Exercise relaxed red-black tree Exercise k 2k 4 Exercise k 1 2k 2 2k 1 Exercise : 1 0

96 96 13 Red-Black Trees 13.2 Rotations Exercise Right-Rotate(T, x) 1 y left[x] Set y. 2 left[x] right[y] Turn y s right subtree into x s left subtree. 3 if right[y] nil 4 then p[right[y]] x 5 p[y] p[x] Link x s parent to y. 6 if p[x] = nil 7 then root[t] y 8 else if left[p[x]] = x 9 then left[p[x]] y 10 else right[p[x]] y 11 right[y] x Put x on y s right. 12 p[x] y Exercise n n 1 n 1 Exercise α β γ a, b, c α, β, γ 13.3 Insertion Exercise Fig Exercise

97 13.4 Deletion 97 Exercise n > 1 n 1 n RB-Insert-Fixup p[z] n 1 n 13.4 Deletion Exercise RB-Delete-Fixup 23 Exercise x p[y] x x Exercise Fig exercise Exercise p[x] w Professors Skelton and Baron The solution to the exercise has to do with two red nodes in a row. The professors are Red Skelton (a comedian) and Red Baron (the WW I flying ace).

98 98 13 Red-Black Trees 13.5 Problems Problem 13 1 Persistent dynamic sets a. b. Presistent-Tree-Insert(T, k) 1 y nil 2 x root[t] 3 while x nil 4 do t Copy(x) 5 if y = nil 6 then r t 7 else if x = left[y] 8 then left[y] = t 9 else right[y] = t 10 y t 11 if key[x] > k 12 then x left[x] 13 else x right[x] 14 if y = nil 15 then return new node with key k. 16 else if key[y] > k 17 then left[y] new node with key k 18 else right[y] new node with key k 19 return r c. O(h) O(h) d., Ω(n) Problem 13 2 Join operation on red-black trees a. RB-Insert RB-Insert-Fixup bh[t] RB-Insert-Fixup z = root[t] bh[t] color[root[t]] black z = root[t] bh[t] bh[t] + 1 RB-Delete RB-Delete-Fixup bh[t] RB-Delete-Fixup 1, 3, 4,bh[T] 2 x x = root[t] bh[t] bh[t] 1 11 x

99 13.5 Problems 99 b. x right[x] bh[x] = bh[t 2 ] O(lg n) c. p[x] p[y],left[x] T y,right[x] T 2 d. color[x] = red RB-Insert-Fixup(T 1, x) O(lg n) e. bh[t 1 ] bh[t 2 ] f. RB-Join O(lg n)

100 Red-Black Trees 13.6 The C implementations of algorithms gcc (GCC) 3.2.3, glibc 2.3.2, GNU gdb 5.3, linux kernel #include <stdio.h> 2 #include <stdlib.h> 3 #include <string.h> 5 /* D e f i n e c o m m a n d s t r i n g l i s t. */ 6 #define INSERT "insert\n" 7 #define PRINT "print\n" 8 #define FIND "find\n" 9 #define DELETE "delete\n" 10 #define CLEAR "clear\n" 11 #define HELP "help\n" 12 #define QUIT "quit\n" 14 /* D e f i n e t h e m a x i m u m l i n e. */ 15 #define MAXLINE typedef int Key; 18 typedef struct red_black_node *rbptr; 19 enum nodecolor {red, black}; 20 typedef struct red_black_node 21 { 22 Key key; 23 rbptr rt; 24 rbptr lft; 25 rbptr pr; 26 enum nodecolor color; 27 } rbnode; 29 static rbnode NIL = {0, NULL, NULL, NULL, black}; 31 rbptr rb_build_tree (rbptr root); 32 rbptr minimum (rbptr x); 33 rbptr maximum (rbptr x); 34 rbptr predecessor (rbptr x); 35 rbptr successor (rbptr x); 36 rbptr rb_search (rbptr x, Key k); 37 rbptr left_rotate (rbptr root, rbptr x); 38 rbptr right_rotate (rbptr root, rbptr x); 39 rbptr rb_insert (rbptr root, rbptr z); 40 rbptr rb_insert_fixup (rbptr root, rbptr z); 41 rbptr rb_delete (rbptr root, rbptr z); 42 rbptr rb_delete_fixup (rbptr root, rbptr z); 43 rbptr rb_clear_tree (rbptr root);

101 45 void rb_print_node (rbptr p); 46 void rb_pre_visit_tree (rbptr root); 47 void usage (void); 49 int getline (char bfu[], int max); 51 rbptr 52 rb_build_tree (rbptr root) 53 { 54 int counter = 1; 55 rbptr p; 56 Key k; 57 char arg[maxline]; 13.6 The C implementations of algorithms printf (">(%d)input key:", counter); 60 while (getline (arg, MAXLINE) > 0) { 61 if (sscanf (arg, "%d", &k) == 1) { 62 if (p = (struct red_black_node *) 63 malloc (sizeof (struct red_black_node))) { 64 p >key = k; 65 p >lft = &NIL; 66 p >rt = &NIL; 67 p >pr = &NIL; 68 p >color = red; 69 root = rb_insert (root, p); 70 ++counter; 71 } 72 printf (">(%d)input key:", counter); 73 } 74 else if (strcmp (arg, "\n") == 0) 75 break; 76 else 77 printf ("?Wrong key type.\n>(%d)input key:", counter); 78 } 79 return root; 80 } 82 rbptr 83 minimum (rbptr x) 84 { 85 while (x >lft!= &NIL) 86 x = x >lft; 87 return x; 88 } 90 rbptr 91 maximum (rbptr x) 92 { 93 while (x >rt!= &NIL)

102 Red-Black Trees 94 x = x >rt; 95 return x; 96 } 98 rbptr 99 successor (rbptr x) 100 { 101 if (x >rt!= &NIL) 102 return (minimum (x >rt)); 103 while (x >pr!= &NIL && x == x >pr >rt) 104 x = x >pr; 105 return x; 106 } 108 rbptr 109 predecessor (rbptr x) 110 { 111 if (x >lft!= &NIL) 112 return (maximum (x)); 113 while (x >pr!= &NIL && x == x >pr >lft) 114 x = x >pr; 115 return x; 116 } 118 rbptr 119 rb_search (rbptr x, Key k) 120 { 121 while (x!= &NIL && x >key!= k) { 122 if (x >key < k) 123 x = x >rt; 124 else 125 x = x >lft; 126 } 127 return x; 128 } 131 rbptr 132 left_rotate (rbptr root, rbptr x) 133 { 134 rbptr r = root; 135 rbptr y; 136 y = x >rt; 138 x >rt = y >lft; 139 y >lft >pr = x; 140 y >pr = x >pr; 141 if (x >pr == &NIL) 142 r = y;