1 The secretary problem

Transcription

1 Thi i new material: if you ee error, pleae jtyu at tanford dot edu 1 The ecretary problem We will tart by analyzing the expected runtime of an algorithm, a you will be expected to do on your homework. Conider the problem of hiring an office aitant. We interview candidate on a rolling bai, and at any given point we want to hire the bet candidate we ve een o far. If a new candidate come along, we immediately fire the old one and hire the new one. Hire-Aitant(n): Randomly huffle the candidate (O(n)) bet = 0 for i = 1 to n: interview candidate i if candidate i i better than candidate bet bet = i hire candidate i In thi model, there i a cot c i aociated with interviewing people, but a much larger cot c h aociated with hiring people. The cot of the algorithm i O(c i n + c h m), where n i the total number of applicant, and m i the number of time we hire a new peron. Exercie: What i the wort-cae cot of thi algorithm? Anwer: In the wort cae cenario, the candidate come in order of increaing quality, and we hire every peron that we interview. Then the hiring cot i O(c h n), and the total cot i O((c i + c h )n). 1.1 Expected hiring cot of HireAitant However, mot of the time, we do not have to hire every candidate, o on average the total cot of the algorithm may be ubtantially le than the wort cae cot. Here we calculate the expected number of time we hire a new candidate Indicator random variable An indicator random variable i a variable that indicate whether an event i happening. If A i an event, then the indicator random variable I A i defined a I A = { 1 if A occur 0 if A doe not occur 1

2 Example: Suppoe we are flipping a coin n time. We let X i be the indicator random variable aociated with the coin coming up head on the ith coin flip. So X i = { 1 if ith coin i head 0 if ith coin i tail By umming the value of X i, we can get the total number of head acro the n coin flip. To find the expected number of head, we firt note that [ n ] n E X i = E[X i ] by linearity of expectation. Now the computation reduce to computing E[X i ] for a ingle coin flip, which i E[X i ] = 1 P (X i = 1) + 0 P (X i = 0) = 1 (1/2) + 0 (1/2) = 1/2 So the expected number of head i n E[X i] = n (1/2) = n/2. Example: Let A be an event and I A be the indicator random variable for that event. Then E[I A ] = P (A). Proof: There are two poibilitie: either A occur, in which cae I A = 1, or A doe not occur, in which cae I A = 0. So E[I A ] = 1 P (A) + 0 P (not A) = P (A) Analyi of HireAitant We are intereted in the number of time we hire a new candidate. Let X i be the indicator random variable that indicate whether candidate i i hired, i.e. let X i = 1 if candidate i i hired, and 0 otherwie. Let X = n X i be the number of time we hire a new candidate. We want to find E[X], which by linearity of expectation, i jut n E[X i]. By the previou example, E[X i ] = P (candidate i i hired), o we jut need to find thi probability. To do thi, we aume that the candidate are interviewed in a random order. (We can enforce thi by including a randomization tep at the beginning of our algorithm.) Candidate i i hired when candidate i i better than all of the candidate 1 through i 1. Now conider only the firt i candidate, which mut appear in a random order. Any one of 2

3 them i equally likely to be the bet-qualified thu far. So the probability that candidate i i better than candidate 1 through i 1 i jut 1/i. Therefore E[X i ] = 1/i, and E[X] = n 1/i = ln n+o(1) (by equation A.7 in the textbook). So the expected number of candidate hired i O(ln n), and the expected hiring cot i O(c h ln n). 2 Lower bound on randomized comparion orting algorithm In lecture, we talked about how any orting algorithm that relie only on comparion between array element for information mut make at leat Ω(n log n) comparion, which give u a lower bound on the runtime. Today we will prove that thi lower bound till hold, even if you ue randomized comparion orting algorithm. A randomized algorithm i jut a program that ue randomne. In Python, you would import the random library. Formally, a randomized algorithm i a program that ha acce to a pecial ource of random bit, and every time it need to make a random deciion, it jut pull the next bit from it ource. A determinitic algorithm i an algorithm that doe not ue randomne (although technically you could ay that a determinitic algorithm i jut a randomized algorithm that ignore it ource of random bit). In thi cla, all the randomized algorithm we ee will produce correct output 100% of the time, but the runtime of the algorithm might vary depending on the reult of the random coin flip. 2.1 Wort cae, determinitic Theorem: Any determinitic comparion-baed orting algorithm mut make at leat Ω(n log n) comparion, in the wort cae. We can view the algorithm a a deciion tree, where node correpond to comparion between array element, and each leaf correpond to an ordering of the array element. 3

4 Now in order for a comparion ort to be correct, the leave of the deciion tree mut contain all poible permutation of the input array. Thi i becaue the input can be in any poible order, o the algorithm mut be able to output all poible ranking of the input element. Indeed, each poible ordering mut be repreented exactly once. That i becaue in order for a leaf to appear on two different deciion tree branche, one of the comparion would have to give different reult for two input, and if the two input are ordered in the ame way, that i impoible. Therefore, there mut be exactly n! leave of the deciion tree (ince there are n! way to order an array of n element). The wort cae runtime i jut the length of the longet path through the deciion tree. Thi i jut Ω(log(n!)). By Stirling formula, thi i Ω(n log n) Why the height of a binary tree with m leave i Ω(log m) If a tree ha height h, it mut have at mot 2 h leave. Thi i becaue each leaf correpond to a unique path from the root of the tree to the bottom of the tree. At each junction, there are at mot two direction the path can take. Since there are at mot h choice to be made, and each choice increae the number of poible path by (at mot) a factor of 2, there are at mot 2 h path to the bottom of the tree, o the tree ha at mot 2 h leave. Therefore, if the tree ha m leave, it height i at leat log m. 4

5 2.2 Average cae, determinitic Average-cae analyi So far thi cla ha mainly focued on the wort cae cot of algorithm, i.e. how fat the algorithm i when run on the wort poible input. Wort cae analyi i very important, but ometime the typical cae i a lot better than the wort cae, o algorithm with poor wort-cae performance may be ueful in practice. Defining the average cae i tricky, becaue it require certain aumption on how often different type of input are fed to the algorithm. If we know the ditribution of input, we can compute the average-cae cot of an algorithm by finding the cot on each input, and then averaging the cot together in accordance with how likely the input i to come up. Example: Suppoe the algorithm cot are a follow: Input Probability that the input happen Cot of the algorithm when run on that input A B C Then the average-cae cot of the algorithm i jut the expected value of the cot, which i Here, it i worth dicuing the difference between expected runtime and average-cae runtime. When we compute the average-cae cot of an algorithm, we are averaging over the different type of input that might be fed to the algorithm. However, when we compute the expected runtime of a randomized algorithm, we are averaging over the inherent randomne in the algorithm. (For example, if our function call random.randint(1, 6), we would run the algorithm once for every poible output of randint(), and average the runtime to find the expected cot) Average cae bound on determinitic comparion-baed orting algorithm So far, we have hown that for any determinitic comparion-baed orting algorithm, we can find an input that force it to make Ω(n log n) comparion. It turn out that we alo have to make Ω(n log n) comparion in the average cae (o it not jut a tatement about uper-pathological input). Remember that thi tatement i meaningle unle we know how often different type of input are fed to the algorithm. When we ay the average cae i Ω(n log n), we aume that each ordering of an input array i equally likely to occur. Under other ditribution of input, thi lower bound doe not necearily hold. Formally, we have 5

6 Theorem: For any determinitic comparion-baed orting algorithm A, the average-cae number of comparion (the number of comparion on average on a randomly choen permutation of n ditinct element) i at leat Ω(log(n!)) = Ω(n log n). Proof: The average-cae number of comparion i imply the average length of the path in the deciion tree from the root to the leave. Thi i becaue each poible permutation correpond to exactly one leaf, which correpond to a unique path in the deciion tree. We find that (1) tree that are more unbalanced have a longer average path length (2) tree that are perfectly balanced have an average path length of log(n!) 1. Together, thee fact how that the average path length will alway be at leat log(n!) Balanced binary tree In a perfectly balanced binary tree with n! leave, the height of each path from the root to a leaf i either the height of the tree, or the height of the tree minu 1. (Let define it that way for now.) Recall that we proved that the height of a tree with n! leave mut be at leat log(n!). (Thi mean it mut be at leat log(n!).) So each path from the root to a leaf mut have length log(n!) 1. Since each input reult in a trip from the root to a leaf, each input mut reult in Ω(log(n!)) comparion, o the average-cae number of comparion i Ω(log(n!)) Unbalanced binary tree In an unbalanced binary tree, the highet leaf x mut be at leat two level higher than the lowet leaf y. In fact, y mut be one of two ibling leave, ince we are dealing with a deciion tree and both deciion mut correpond to an output. Moving y and y ibling to be the child of x decreae the average height of the leave. If the maller depth i d and the larger depth i D, then we have removed two leave of depth D (y and y ibling) and one leaf of depth d (namely x), and we have added two leave of depth d + 1 (y and y ibling) and one leaf of depth D 1 (y parent). Therefore, any unbalanced deciion tree can be modified to have maller average depth, o the deciion tree with the lowet average depth mut be balanced. Therefore, the average-cae number of comparion i till Ω(log(n!)). 2.3 Average cae, randomized You may think of a randomized algorithm a a collection of determinitic algorithm, each with the random number choice hardwired in. For example, if your algorithm A involve chooing a random number between 1 and 6, you can actually repreent it a ix different determinitic algorithm A 1,..., A, where algorithm A i the verion of A where we choe i a our random number. 6

7 Then the expected number of comparion made by randomized algorithm A on an input I i jut P r()(number of comparion of A on I) Now to find the average cae number of comparion, we mut average thi runtime over all poible input I: P r(i) P r()(number of comparion of A on I) I We can revere the order of ummation: P r() P r(i)(number of comparion of A on I) I And ince any determinitic algorithm make at leat log(n!) 1 comparion in the average cae, our current expreion mut be at leat P r()( log(n!) 1) And ince P r() = 1, thi i jut log(n!) 1 = Ω(n log n). 2.4 Wort cae, randomized Wort cae lower bound are alway at leat a large a average cae lower bound, o in the wort cae, we alo get Ω(n log n) comparion. 3 Reference Comparion-baed Lower Bound For Sorting 7