Math 216A Notes, Week 4

Transcription

1 Math 216A Notes, Week 4 Scribe: Jaso Erbele Disclaimer: These otes are ot early as polished ad quite possibly ot early as correct) as a published paper. Please use them at your ow risk. 1. Probabilistic Methods Two obvious facts that we will use over ad over agai are the followig: If a evet has positive probability of occurrig, it s possible for it to occur. If the expectatio of a radom variable, X, is EX), the value of X will sometimes be at least EX) ad sometimes at most EX). A immediate corollary to the first obvious fact is that if a evet has probability less tha 1 of occurrig, it s possible for it to ot occur. The goal whe lookig at probabilistic methods is to use these two facts i every way possible. The first example we will cosider is the questio of the existece of k-balaced touramets Balaced Touramets. Defiitio 1. A touramet is a directed graph o some vertex set, V, so that for each pair of x, y V, exactly oe directed edge exists betwee them. This ca be thought of as a roud-robi touramet, where every player plays every other player, ad there are o draws. Below are two examples of touramets o three vertices, or three players. Defiitio 2. A touramet is k-balaced if for every subset S of V with S = k, there is a vertex, x, such that all the edges from S to x are directed away from x. This ca be thought of as for ay give set of k players, there is always at least oe player that beats all of them. It ca be quickly see from the defiitio that a k-balaced touramet will also be k 1-balaced. I the above examples, the first touramet is ot 1-balaced because there is a player that o other player beats. The secod touramet, however, is 1-balaced, but it is ot 2-balaced. Usig stadard techiques, the questio of whether a k-balaced touramet ca exist would come to a lot of casework ad/or a clever usage of symmetry to costruct ad the check. If we take V = = the umber of players it is possible to show the existece of k-balaced touramets for ay give k, provided is large eough. The idea is to costruct the touramet, T, at radom ad show the probability it is k-balaced is positive. First we eed to decide how to radomly costruct a touramet. Let x, y) deote the edge directed from x to y, i.e. x beats y, for x, y V. For every pair, {x, y}, of vertices, let the edge be directed x, y) with probability 1 2, ad directed y, x) with probability 1 2. Each pair, or each game, is thus idepedet of the others. I the three vertex touramets above, there is probability 1 8 of either of them happeig, as there are eight equally likely possibilities for directig the edges. Istead of cosiderig a lower boud o the probability of a k-balaced touramet directly, cosider a upper boud o the probability of a touramet beig ot k-balaced. For a set, S, with S = k, we say it is bad if there is o player that beats everyoe i S. Abusig otatio, we refer to this as there beig o vertex with x, S). For idividual vertices, the probability of this happeig is 1 2 k. What makes this work 1

2 icely is for x, y / S, x, S) ad y, S) are idepedet evets, as there are o shared edges. So to further abuse otatio, p, S)) = k ) k. The probability that the touramet is ot k-balaced is the probability that oe of these obody beats S evets occur. These evets are o loger idepedet, so we will use a powerful tool to estimate this combied probability called the Uio Boud. Theorem 1. Uio Boud) If B 1,..., B m are ay evets, P m i=1 B i) PB 1 ) + + PB m ). Equality occurs whe all the evets are o-overlappig. Overlap causes the right had side to be a potetially extreme overestimate. I this case the B i s are the obody beats a give S evets, of which there are ) k. Thus the probability of at least oe of these evets occurrig is at most ) k 1 1 ) k. If this upper 2 k boud is smaller tha 1, there must be a k-balaced touramet o vertices. A cheap boud o k) is ) k k. We ca do better tha this, ad we may eed to later o, but this will suffice for ow. Thus the probability that at least oe set S has obody defeatig it is at most k 1 1 ) k, which teds to 0 as 2 k. So for large eough, the probability of a touramet ot beig k-balaced is smaller tha 1, which meas k-balaced touramets exist. This example ca be summarized as follows: to show there are these kids of evely spread out touramets, assig the edges radomly. Next, put a upper boud o the probability that a touramet is bad, usig the Uio Boud. If the upper boud is less tha 1, we re doe MaxCut, ad liearity of expectatio. Here s aother result that ca quickly be proved by this sort of radomess argumet. Theorem 2. For ay graph G, there is a partitio of the vertices ito two subsets A ad B such that at least half of the edges of G have their edpoits i differet sets. I other words, ay graph has cut at least half the size of the edge set of the graph. As a ote, this 1 2 is sharp: the complete graph o vertices has roughly edges. Techically, 1) 2, but that s about 2 2. If the sets are split ito size α ad 1 α), the α1 α) 2 edges cross. This is largest whe α = 1 2 ad you get edges cut. The best we ca do for the complete graph is to split everythig evely. There is othig i the theorem that specifies aythig about the graph, so with o kowledge at all about G the first thig we might try is just splittig radomly ad hopig. More specifically, put each vertex idepedetly i A with probability 1 2, ad put it i B with probability 1 2. Give a edge, the probability that the edpoits ed up i differet sets is 1 2. Agai, a ufortuate thig is that although our basic edge probabilities are idepedet of each other, the probabilities of iterest are ot. E.g., for each edge of a triagle there is 1 chace i 2 that it splits, but there is o way to split all three at oce. To actually compute the probability that exactly k edges are split would be ugly. As before, there s a basic fact from probability to save the day. This oe is called Liearity of Expectatio. Theorem 3. For ay two radom variables, X 1 ad X 2, EX 1 + X 2 ) = EX 1 ) + EX 2 ). Notice that there are o restrictios o X 1 ad X 2. They ca be depedet, idepedet, whatever. The sophisticated-lookig proof of this from measure theory is essetially that the itegral of a sum is the sum of the itegrals, sice a expectatio, or the average value, of a variable is the itegral of that variable with respect to the probability measure. { 1 the edge is splits So here what we re goig to do is we re goig to say, for ay edge, let fe) = 0 else 2.

3 We kow that Efe)) = 1 2. So by liearity of expectatio, the expected umber of edges cut by a radom cut is 1 2 e. Ad if the average split cuts half the edges, there must be a split that cuts half the edges Coditioal Expectatios. As it turs out, probability is t really ecessary here. What we could have doe istead was a sort of greedy algorithm: Place the vertices i a arbitrary order {v 1,..., v }, ad the place each v i oe by oe. At each step, always place v i i the set which cuts more of the edges betwee v i ad {v 1,..., v i 1 }. This is a special case of what is termed a coditioal expectatio argumet. The idea is as follows: Imagie the radom process as a sort of decisio tree. First we decide whether to place vertex 1 i A or B, the vertex 2 i A or B, ad so o. If all the steps are completely radom, the o average we cut half the edges. Now every time we have to make a decisio, we always choose to move dow the brach of the tree with higher coditioal expectatio give what we ve doe so far, assumig all future decisios are radom. Example: Suppose that vertex v 1 has already bee placed i A, ad vertex v 2 the secod to be placed) is adjacet to vertex 1. If we place v 2 i A, the the expected umber of cut edges is e 1 2 all edges but the v 1 v 2 edge stad a 1/2 chace of beig cut, but the v 1 v 2 edge is ucut). If we place v 2 i B, the expected umber of cut edges if e+1 2 same as before, but ow v 1 v 2 is cut), so we place v 2 i B. I geeral, the coditioal expectatios argumet behaves exactly like the greedy algorithm here. The poit here is that the expectatio was iitially half the edges, ad ca oly icrease each time we make a decisio we always move to the better brach). At the ed of the algorithm, the coditioal expectatio exactly equals the umber of edges cut there s o radomess left), so we must have cut half the edges. I theory, the same deradomizatio works for the radom touramet example, but there s a catch. Say I m tryig to fid a 50-balaced touramet. The to compute the coditioal expectatio I eed to use to make my decisio, I eed to keep track of 50) differet sets, which is hard computatioally. 2. Domiatig Sets So our philosophy so far has I guess bee alog the lies of, okay, if I wat to make the set work, choose a radom set, see if it works. If I wat to see if a touramet works, choose a radom touramet, see if it works. Let s look at somethig that maybe a little bit more complicated. Agai we re goig to start with a defiitio. Defiitio 3. Give a graph, G, we call a subset, S, of vertices domiatig if every vertex o the graph is either i S or adjacet to somethig i S. Ituitively, thik about S as sort of beig here a set of sesors. A sesor ca oly see so far; it ca oly see thigs adjacet to it. A domiatig set is a set of sesors that ca see everythig o the graph. Sesors are expesive; small domiatig sets are good. I mea, if I took every vertex i the graph, that would be a domiatig set, but that would be a expesive oe. Now maybe, okay maybe i some cases I do eed to take every vertex. If my origial graph was the empty graph, the I ca t do aythig better tha just takig every vertex. But somehow the feelig is, if the graph has a lot of edges, if all the vertices have a lot of eighbors, we should be able to do better. So a questio. If we re give a d, ad we kow that G is d-regular o vertices, but I do t kow aythig about G other tha that it s d-regular. I wat to kow what ca I say about the smallest domiatig set. So here s a quick lower boud. S better be at least d+1. To see this, let s say I place my sesors oe by oe. I place my first sesor dow, mark my first vertex i the graph, ad it ca see exactly d + 1 vertices. Let s 3

4 use better laguage: So far d + 1 vertices have bee covered. After two sesors have bee placed, there s at most 2d + 1) covered vertices. Not equal aymore, because maybe some of those d + 1 vertices were already see by the first sesor. I the absolute best case for us, we still oly got 2d + 1) vertices covered. So if I place k sesors, there s at most kd + 1) covered, ad I oly stop whe vertices covered. So k had better be at least d+1. [After each placemet, at most d + 1 ew vertices are covered.] But i geeral, I m ot goig to be able to get d+1, because that s oly if I ca miraculously cover the graph perfectly. Ad I do t kow if I ca cover the graph perfectly. I do t kow aythig at all about G. But it s sort of the same way as before: I wat my sesors sort of spread out over the graph so they do t overlap too much. So maybe I should try placig the sesors at radom. There s a couple of ways you ca thik about placig the sesors at radom. You ca thik about it as sayig, I m goig to pick k vertices o the graph, uiformly over all sets of size k. That s a little aoyig, actually, to work with because there is ot much idepedece whe we do it that way if I kow that vertex x got a sesor, that makes it a tiy bit less likely that each other vertex gets a sesor). Ad I guess sort of a ituitio is that whe i doubt, try to maximize idepedece. It makes computatios a lot icer. So my radom sesor placemet, or my first try at radom sesor placemet, I m goig to place a sesor at each vertex with probability p. Idepedetly. I have t said what p is. p is somethig I m goig to pick later o i the argumet, oce I see what the bouds look like i terms of p. Or if you wat it i terms of the origial laguage, probability that x S is p for every x. So give ay x the probability that x is ucovered is, just 1 p) d+1 because I ve got my x here, I ve got my d eighbors, ad the oly time that x is ucovered is if all d + 1 of these vertices were ot chose this sort of calculatio is why you maximize idepedece!). The first thought might be to choose p so that with pretty high probability othig is ucovered. But there s a problem here. The problem is that if is large, I eed p pretty close to 1 before these evets are all ulikely. If d is small ad >> d, the I ve got a buch of areas of the graph where I could fail. Ad if every area of the graph fails with probability 1% ad there s eough areas of the graph, I m probably goig to fail. So if I wat to make sure I do t fail, p had better be really close to 1. But if p is really close to 1, I ve essetially picked everythig. Ad from a efficiecy poit of view, that s awful. So this sort of simple pick a set at radom, hope it works, is ot the way to go about here. But maybe there s a way we ca improve this. Suppose that d is large ad p ot too small. The most vertices are probably goig to be covered the probability that ay vertex is ucovered is small). So say I ve already covered 99.9% of the vertices i my graph. At this poit I do t wat to pick vertices at radom aymore, because a radomly placed sesor stads a good chace of missig all of the vertices I still have left to cover. There s o poit i placig a sesor where I ca already see everythig! So istead, let s do a two-step process. Step 1 is goig to be the same as before. I m goig to place each vertex i S with probability p. Ad agai, I still have t decided what p is yet. At this poit, we still have some subset S of the vertices which is ucovered. For step 2, I ll just be lazy ad stick all of S i S. Step 2 is ot perfectly efficiet, if you thik about it. It might be that I have two adjacet vertices, ad I stick both of them i whe I could have just used oe. But we already did t kow much about G beyod its regularity, ad with all the radomess we ve doe already we really have almost o uderstadig of what S looks like ad what vertices may or may ot be adjacet. Besides, if we pick p right the S will oly be a very tiy set. Ad if we oly have 1% of the graph left, who cares if we re ot too efficiet o that tiy fractio? I just wat to make sure I have a domiatig set. Ad by costructio, we ve foud oe. The real questio is, is it small? Well let s see. The expected size of S after step 1 is just p. vertices, I iclude each oe with probability p. I step 2, well, for ay give vertex, the probability I eed to add that vertex to S i step 2 is just: 1 p) d+1. 4

5 So the expected umber of added vertices is 1 p) d+1. By liearity of expectatio, the expected size of S after step 2 is p + 1 p) d+1 ). The key thig here is that, agai, there must be a domiatig set at least as small as the average from our costructio. Ad this is true o matter what p is. So you should thik about this as sort of a large family of bouds o the size of the smallest domiatig set. We ca just choose the value of p that gives us the best boud. Here s a boud which is t the tightest possible, but looks ice ad is pretty close for large d. What we use is the basic calculus idetity 1 + x e x This boud will show up a LOT i the future, because it s so useful i hadlig expressios like 1 p) d+1. I our case it tells us E S ) p + e pd+1) ) To optimize over p, I take the derivative, 1 d + 1)e pd+1) = 0. Solve it, you get p = ld+1) d+1. Pluggig this i to our upper boud, we get that every d regular graph has a domiatig set of size at most 1+ ld+1) d+1 ). As it turs out, for large d I believe this boud is essetially the correct boud up to a costat factor at worst). I have t actually see the full proof of this, but there is a referece to it give i the Alo ad Specer Probabilistic Methods book that I liked o the ilear. So questios o ay of this? Questio: Is there a deradomizatio method where you kid of order the vertices by their valecies ad you start pickig them up i that order? Aswer: Probably. I d have to actually work through what would happe, but it s essetially a questio questio of how large the expected valecy is at each step. Sooer or later you re ot goig to be able to pick vertices of valecy d + 1 ay more, but if the average valecy of the graph is large eough, the, the boud might work. 3. Sum-Free Sets Here s oe more example of this sort of just pick the right probability space ad everythig pops out type of proof. Here, actually, the big trick is figurig out what probability space to work with. Oce agai we start with a defiitio Defiitio 4. A set A is sum-free if there are o solutios to x + y = z with x, y, z A. So, examples of sum-free sets are the odd umbers. Or all all the umbers from 11 up to 21. So a questio - I m ot sure who origially asked it - give a set B, does it ecessarily cotai I m goig to be vague here) a large sum-free A? So you give me a B, I eed to fid A i it that is both large ad sum-free. Theorem 4. Erdős, 1965) Ay subset B of itegers cotais a sum-free A with A > B 3. For may B you ca fid sum-free subsets which are much larger. For example, if half the umbers i B are odd, ad I ll just say, take the odd umbers ad be doe. But we do t kow aythig about B here. The proof sort of relies o two really ifty tricks. Oe trick is to work i, well, I m ot goig to work i the itegers ay more, I m goig to work I m goig to work i the itegers mod p, where p is prime ad p, the largest member of B. It s harder to be sum-free mod p tha it is to be sum-free i geeral. So if I ca fid a sum-free set mod p, I ca certaily fid a sum-free set i geeral. Ad the idea is that i Z p, there is oe very large sum-free set. Specifically, if I take the x such that p 3 x < 2p 3. Because if I take two thigs i the lower part of this set, the sum is bigger tha 2p 3, ad if I take 5

6 two thigs i the upper part, the sum is t large eough to wrap back aroud to p 3. So if 1 3 of the umbers i my set happe to fall i this rage, I m doe. But there s o reaso to expect me to be that lucky. Here s where the secod trick comes i. What we re goig to do is we re goig to look at the set m B, where m is radom i Z p here the otatio m B meas take each elemet of B ad multiply it by m). So it s sort of a completely differet probability space to what we ve see so far. Istead of takig radom thigs i B to put i A, I m takig B ad I m multiplyig it by a radom umber. Ad I m goig to say, let S m be the x B such that p 3 mx < 2p 3 mod p. My claim is that S m is sum-free, because if x, y, z S m ad x+y = z, well that tells me that mx+my = mz. Ad mx, my, ad mz are all i this p 3 to 2p 3 rage. So I have a whole collectio of these S ms, ad if I ca show that ay of them are large, I m doe. Let x i,m equal 1 if i S m, ad 0 otherwise. We have S m = m m i B = i B m = i B p 3 x i,m x i,m = B p 3 There are p 1 terms i the sum, so some S m must have size larger tha B /3. Essetially the hard part is comig up with the idea of workig i a field ad the set S m. 4. Splittig Vectors Evely To cotiue with the idea of costructig a distributio with a small/large expectatio to show a variable ca be made small/large, let s cosider aother example: Give uit vectors, x 1,..., x i R m, ca we divide the x i ito two sets, A ad B so that x x i A i x x i B i? More precisely, we wish to miimize the Euclidea orm, x x i A i x x i B i. Algorithmically this is a very hard problem, eve if x 1,..., x are just real umbers. So istead of tryig to get the best possible boud for each set, let s try ad get some sort of upper boud which holds for all sets. A bad example, i the sese that there is o good split, would be takig m > ad x 1,..., x mutually orthogoal. By the Pythagorea theorem, x x i A i x x i B i =. So kowig othig about the dimesio of the space or directios the vectors poit, there are some cases where the miimum is at least. Claim 1. There is always a split such that x x i A i x x i B i Together with what we have already observed, this would give a sharp upper boud o the miimum distace. To prove this we will cosider a radom partitio with each vector assiged idepedetly to A or B with probabilities px i A) = px i B) = 1 2. The x x i A i x x i B i = i=1 ɛ ix i, where ɛ i { 1, 1} with equal probability of beig 1 or 1. The idea agai is that if we ca show this sum is o average small, there must be a partitio which makes it small. However, it is very icoveiet to work directly with the expectatio because the square roots get ugly to deal with. To compesate, let Y = i=1 ɛ ix i 2. It s eough 6

7 to show that EY ) is small, ad ow we have EY ) = E ɛ i x i, ɛ i x i ) = E ɛ i ɛ j x i, x j ) i j = Eɛ i ɛ j ) x i, x j i j = 1 x i, x i =. i Here we used that ier products ad expectatios are both liear, that { 0 i j Eɛ i ɛ j ) = 1 i = j by the idepedece of ɛ i ad ɛ 2 i = 1, ad that x i is a uit vector. No matter what the vectors are, the expected square of the distace is the same:. I the oe extreme whe all the vectors are orthogoal, the distace is always. I the other extreme, if all vectors are i the same directio, this is sayig that the expected square distace of a radom walk from 0 after steps is. So there must be a partitio where the square distace is at most this large. The tricks here ivolved buildig a radom partitio, squarig the distaces, ad exploitig the fact that the expectatio of a squared sum is the expectatio of a double sum, from which we ca use liearity to make thigs work icely. 5. The First Momet Method Markov s Iequality) ad Balls i Bis Aother way to use probabilistic methods is to cosider a radom process istead of isertig radomess ito the calculatios of determiistic processes. Now we wat to uderstad how the process usually behaves, i the sese that ofte there is a parameter that the probability might deped o. I the above example, the umber of vectors could be icreased, perhaps. Our hope is that the radomess becomes less radom i the limit. A evet is said to happe almost surely if its probability approaches 1. If a coi is flipped times, almost surely there will be at least oe time it lads o heads. Assumig the law of large umbers, we could also say that almost surely there will be aroud 2 heads. Maybe for some radom variable, x, oe might wish to show x fx) 1 almost surely. Takig x to be the umber of heads, ad fx) = 2, we might try to show this happes as teds to, or at least that it s ulikely for that ratio to be larger tha 1.5 as. Ad the how ulikely is ulikely? Example: Give m balls, bis, large, where each ball is tossed idepedetly ad radomly ito a bi that is, pball i i bi j) = 1 ). There are may thigs we might cosider. For example, you might care about load-balacig issues: How may balls are i the most crowded bi? For this example we will istead cosider the coupo collector s problem : how likely is it that all the bis have a ball? This will clearly deped o m, as m < immediately meas there will certaily be some empty bis, whereas if m >>, it is ituitively very ulikely there will be ay empty bis. There should be a boudary betwee these extremes, so to reword the questio, how large does m have to be before it becomes likely all bis are full? A easier questio is, what is pbi 1 empty)? pbi 1 empty) = 1 1 )m. 7

8 Let X be the umber of empty bis. { 1 if bi i empty X = x x : x i = 0 else Thus EX) = i=1 Ex i) = 1 1 ) m. We ca asymptotically aalyse this expectatio to see whe it is large or small, but we eed to be able to say two thigs before we ca traslate from the asymptotics o EX) to asymptotics o X. Recall that what we really care about is px = 0). If EX) is small, does that mea X = 0, or at least is small most of the time? If EX) is large, does that mea X is large most of the time?. A example that was used durig to Cold War to illustrate this difficulty was, let X equal the umber of people dead i the ext decade due to uclear war. px = 0) is very close to 1 due to the very low probability of uclear war actually breakig out, but EX) could be fairly sigificat due to the very large umber of people that would die to uclear war if uclear war did happe. So EX) ca be large despite X beig small most of the time. This uclear war example shows EX) large X large, however, EX) small X small if we add oe extra criterio. Theorem 5. Markov s Iequality: If X is a o-egative radom variable, the px λex)) 1 λ, or with a slight chage of variables, px λ) EX) λ. The o-egative assumptio is ecessary, as there ca be big positive ad egative values that cacel out if that assumptio is ot made. The ɛ i idicator variables i the previous example demostrate that cocept: Eɛ i ) = 0, ad Markov s iequality clearly fails. Proof. Defie a ew variable, Y, where Y = { λ if X λ 0 else By the costructio of Y, X Y, so it follows that EX) EY ) = λ px λ). Because λ > 0, it is trivial to obtai the secod form of Markov s iequality.. A special case that frequetly shows up i combiatorics is whe X is a o-egative iteger-valued radom variable, i which case px > 0) = px 1). Applyig Markov s iequality, we get px > 0) EX). If a radom variable ca be defied i such a way that its expectatio teds to zero, the variable will almost always equal zero, if it is iteger-valued. Returig to the balls ad bis, whe will the expectatio for at least oe empty bi, foud above, ted to zero? If we take m = l + t, where t is a as yet udetermied variable, the EX) = 1 ) 1 l +t e l +t)/ = e t/, where we used the iequality 1 x e x. So if m = l + t ad t Pall bis have a ball) 1., the EX) 0, so The short versio of this argumet is, fid the expected umber of empty bis ad determie what m has to be for the expected umber of empty bis to ted to zero. That gives a upper boud for how large m has to be before the bis are almost surely full. The other half of what we wat to argue here is that if 8

9 m << l, the umber of empty bis is probably bigger tha zero. This is the implicatio that iitially is ot true just by lookig at the expectatio. It is true that the expected umber of empty bis teds to, but a argumet is still eeded to show that the umber of empty bis is close to the expectatio: that it is ot 0 with large probability, yet huge with tiy probability. The usual trick used to make this argumet is cosiderig the secod momet istead of just the first momet. I the laguage of statistics, the mea o loger suffices, so we cosider the variace. 6. The Secod Momet, ad Chebyshev s Iequality We defie the Variace VarX) of a variable X to be VarX) = EX EX)) 2 ) = EX 2 ) EX) 2. Other otatio frequetly used for variace ad expectatio is VarX) := σ 2 ad EX) = µ. The ituitio is that if the variace is small, x is usually close to its expectatio, Ex). A way to make this ituitio precise is by usig aother iequality, called Chebyshev s Iequality. It basically says that most of the time a radom variable will be withi a small umber of stadard deviatios of the mea. Theorem 6. Chebyshev s Iequality:) P X µ λσ) 1 λ 2 Proof. Let Y = X µ) 2. The PX µ) λσ) = Px µ) 2 λ 2 σ 2 ) usig the same sort of trick as above with the distaces squared. By Markov s iequality, applied to Y, Px µ) 2 λ 2 σ 2 ) Ex µ)2 ) λ 2 σ = 1 2 λ, 2 sice Ex µ) 2 ) = Ex Ex)) 2 ) = Varx) = σ 2. A special case of Chebyshev s iequality that shows up frequetly i practice: if σ << µ, the with high probability X µ. More precisely, if σ µ 0, the P1 ɛ)µ X 1 + ɛ)µ) 1 for ay ɛ. We ca see this by pluggig ɛµ ito Chebyshev s iequality ad otig that λ teds to, so the probability it is ot i this rage teds to 0. To show a radom variable is usually large, oe way is to first show the expectatio is large, ad the show the variace is small. That meas the variable is usually close to its expectatio, which was already show to be large. Ufortuately, as ca be see below, secod momet calculatios are usually a bit messier tha first momet calculatios. 7. Back to Balls i Bis Returig to balls ad bis, where we have the same radom variable as before ad the same expectatio as before, but ow we care about the expectatio of X 2. The followig is oe way to calculate EX 2 ) that may ot be the most efficiet, but is the kid of calculatio that shows up frequetly. 9

10 [ ] 2 EX EX)) 2 = E x i Ex i ) by liearity) = E i = i=1 [x i Ex i )][x j Ex j )] j i=1 E[x i Ex i )] 2 ) + i j E[x i Ex i )][x j Ex j )]) = E[x 1 Ex 1 )] 2 ) + 2 ) E[x 1 Ex 1 )][x 2 Ex 2 )]), by symmetry. Although this keeps gettig messier, we eed to keep i mid that we do t care about the exact value of the variace so much as a upper boud that shows it is much smaller tha the mea. With that i mid, cosider the two terms separately, ad boud each oe of them. I the first term, Ex i Ex i )) 2 ) = Varx 1 ). Because x 1 is a idicator variable, it is either 0 or 1, ad x 1 = 1 with probability p = 1 1 )m, so Varx 1 ) = p1 p) p. This upper boud should be close to the actual amout wheever p is close to zero. Thus the first term is at most 1 1 )m. For the secod term, Ex 1 Ex 1 ))x 2 Ex 2 ))) = Ex 1 x 2 x 1 Ex 2 ) Ex 1 )x 2 + Ex 1 )Ex 2 )) = Ex 1 x 2 ) Ex 1 Ex 2 )) EEx 1 )x 2 ) + EEx 1 )Ex 2 )) = Ex 1 x 2 ) Ex 1 )Ex 2 ) Ex 1 )Ex 2 ) + Ex 1 )Ex 2 ) = Ex 1 x 2 ) Ex 1 )Ex 2 ), where Ex 1 x 2 ) = Pbis 1,2 empty) = 1 2 ) m. So Ex1 x 2 ) Ex 1 )Ex 2 ) = 1 2 ) m 1 1 ) 2m. We caot use our usual asymptotic trick of replacig each of these with expoetials, sice they will be the same expoetial i both cases. Cosider, the, the lower order terms: Ex 1 x 2 ) Ex 1 )Ex 2 ) = 1 ) 2 m ) m ) m k ) k ) 1 m 2. k k 1 This is a upper boud because aside from the terms that cacelled immediately, all the terms are take here to be positive. Sice m k ) m k, the covariace is at most k ) m k m ) k. 2 We kow what rage of m to cosider, sice if m > l we already kow there will probably be o empty bis. If m l, this sum is domiated by the k = 1 term, so the sum is 1 + o1)) m 2 )1 2 )m. Combiig everythig, the first term is at most 1 1 )m, ad the secod term is bouded by m )m. The whole thig is V arx) 1 1 ) m + 2 ) m ) m 1 + o1)) e m/ + me 2m/ 1 + o1)). 10

11 Sice the mea is e m/, we ca ow apply Chebyshev s iequality: px = 0) p X µ µ) σ2 µ 2 e m/ + me 2m/ = em/ 2 e 2m/ + m 2. We take m = l t, so P X = 0) which teds to 0 as t teds to ifiity. I summary, by the first momet, if we take t >> ad put l + t balls i bis: po empty bis) 1. t/ el + l 2 By the secod momet, if we put l t balls i bis: po empty bis) 0. = e t/ + l, I other words, there is a sharp threshold. If the balls are placed i bis oe at a time, we expect the last bi to be filled to get its first ball after about l balls. That is, radomly droppig the balls costs the efficiecy a factor of l. A similar argumet ca be made to show the threshold for the umber of balls i the most full bi whe fillig bis radomly with balls is about log log log. 1 The idea to take away here is this two-stage argumet. We re tryig to fid the threshold for m where the bis became full. Oe side is really easy; look at the first momet ad show the expectatio is small. The other side was a bit of a pai. We looked at the expectatio of the square ad showed that it was ot too big. I practice, sometimes the expectatio of the square is really log ad captures all of the work. You ca fid page papers where oe page has the first momet calculatios, ad the rest is to estimate the variace. Next week we will look at a few more examples, ad the look for a better method. Chebyshev s iequality oly falls off as 1 λ 2, whereas somethig like the ormal distributio falls off expoetially. We should be able to get that expoetial kid of tail if thigs behave icely eough. 1 There is a iterestig tradeoff i efficiecy whe fillig bis radomly versus checkig each bi for a empty bi to fill. The latter ivolves checkig a large umber of bis every time a ew ball arrives to see which bi has the fewest balls), while the former leads to a slow, but growig) cluster of balls i the same bi. A itermediate kid of bi fillig protocol would be to have each ball choose two bis radomly, goig ito the least full of the two. With that protocol, the most full bi is expected to oly have about l l balls i it. For a fuller overview of this idea that two choices dramatically improves efficiecy, take a look at a thig called The Power of Two Choices. 11