Lecture 5: Probablty Dstrbutons Random Varables Probablty Dstrbutons Dscrete Random Varables Contnuous Random Varables and ther Dstrbutons Dscrete Jont Dstrbutons Contnuous Jont Dstrbutons Independent d Random Varables Summary Measures Moments of Condtonal and Jont Dstrbutons Correlaton and Covarance Random Varables A sample space s a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the real lne. It s gven by x : S R. Each element n the sample space has an assocated probablty and such probabltes sum or ntegrate to one. 1
Probablty Dstrbutons Let A R and let Prob(x A) denote the probablty blt that t x wll belong to A. Def. The dstrbuton functon of a random varable x s a functon defned by F(x') Prob(x x'), x' R. Key Propertes P.1 F s nondecreasng n x. P.2 lm F(x) = 1 and lm F(x) = 0. x x P.3 F s contnuous from the rght. P4 P.4 Forallx' x', Prob(x>x')=1 x') - F(x'). 2
Dscrete Random Varables If the random varable can assume only a fnte t number or a countable nfnte t set of values, t s sad to be a dscrete random varable. Key Propertes P.1 Prob(x = x') f(x') 0. (f s called the probablty mass functon or the probablty functon.) P.2 f( x ) Pr ob( x x ) 1. 1 1 P.3 Prob(x A) = f( x ). x A 3
Examples Example: #1 Consder the random varable assocated wth 2 tosses of a far con. The possble values for the #heads x are {0, 1, 2}. We have that f(0) = 1/4, f(1) = 1/2, and f(2) = 1/4. f(x) F(x) 1 X 1/2 X 3/4 X 1/4 X X 1/4 X 0 1 2 0 1 2 Examples #2 A sngle toss of a far de. f(x) = 1/6 f x = 1,2,3,4,5,6. F(x ) = x /6. 4
Contnuous Random Varables and ther Dstrbutons Def. A random varable x has a contnuous dstrbuton f there exsts a nonnegatve functon f defned on R such that for any nterval A of R Prob (x A) = f() x dx. xa The functon f s called the probablty densty functon of x and the doman of f s called the support of the random varable x. Propertes of f P.1 f(x) 0, for all x. P.2 f ( x) dx 1. P.3 If df/dx exsts, then df/dx = f(x), for all x. In terms of geometry F(x) s the area under f(x) for x' x. 5
Example Example: The unform dstrbuton on [a,b]. 1/(b-a), f x [a,b] f(x) = 0, otherwse Note that F s gven by x 1 1 F(x) = [ 1 / ( b a)] dx. ( b a) x x a ( b a) ( b a) x a a Also, b b 1 a f( x) dx [ 1 / ( b a)] dx 1. ( b a) x b a b a a ( b a) ( b a) F(x) Example 1 slope =1/(b-a) -a/(b-a) a b x f(x) 1/(b-a) a b x 6
Dscrete Jont Dstrbutons Let the two random varables x and y have a jont probablty blt functon f(x ',, y ') = Prob(x = x ' and y = y '). Propertes of Prob Functon P.1 f(x, y ) 0. P.2 Prob((x,y ) A) = f ( x, y ). P.3 f( x, y ) = 1. ( x, y ) ( x, y ) A 7
The Dstrbuton Functon Defned F(x ', y ' ' ) = Prob( x x and y y ' ) = f ( x, y ), where ' L = {(x, y ) : x x and y y ' }. ( x, y ) L Margnal Prob and Dstrbuton Functons The margnal probablty functon assocated wth x s gven by f 1 (x j ) Prob(x = x j ) = f ( x, y ) y The margnal probablty functon assocated wth y s gven by f 2 (y j ) Prob(y = y j ) = f ( x, y ) x j j 8
Margnal dstrbuton functons The margnal dstrbuton functon of x s gven by F 1 (x j ) = Prob(x x j ) = lm yj Prob(x x j and y y j ) = lm yj F(x j,y j ). Lkewse for y, the margnal dstrbuton functon s F 2 (y j ) = lm xj F(x j,y j ). Example An example. Let x and y represent random varables representng whether or not two dfferent stocks wll ncrease or decrease n prce. Each of x and y can take on the values 0 or 1, where a 1 means that ts prce has ncreased and a 0 means that t has decreased. The probablty functon s descrbed by f(1,1) =.50 f(0,1) =.35 f(1,0) =.10 f(0,0) =.05. Answer each of the followng questons. a. Fnd F(1,0) and F(0,1). F(1,0) =.1 +.05 =.15. F(0,1) =.35 +.05 =.40. b. Fnd F 1 (0) = lm F(0,y) = F(0,1) =.4 y 1 c. Fnd F 2 (1) = lm F(x,1) = F(1,1) = 1. x 1 d. Fnd f 1 (0) = f( 0, y) f(0,1) + f(0,0) =.4. y e. Fnd f 1 (1) = f(, 1 y) f(1,1) +f(1,0) =.6 y 9
Condtonal Dstrbutons After a value of y has been observed, the probablty blt that t a value of x wll be observed s gven by Prob(x = x y = y ) = Pr ob ( x x & y y ). Pr ob( y y ) The functon g 1 (x y ) fx (, y ) f2 ( y ). s called the condtonal probablty functon of x, gven y. g 2 (y x ) s defned analogously. Propertes of Condtonal Probablty Functons () g 1 (x y ) 0. () g 1 (x y ) = x f(x,y ) / x (() and () hold for g 2 (y x )) f(x,y ) = 1. x () f(x,y ) = g 1 (x y )f 2 (y ) = g 2 (y x )f 1 (x ). 10
Condtonal Dstrbuton Functons F 1 (x y ) = fx (, y)/ f2 ( y), x x F 2 (y x ) = fx (, y)/ f1( x). y y The stock prce example revsted a. Compute g 1 (1 0) = f(1,0)/f 2 (0). We have that f 2 (0) = f(0,0) 0) + f(1,0) =.05 +.1 =.15. Further f(1,0) =.1. Thus, g 1 (1 0) =.1/.15 =.66. b. Fnd g 2 (0 0) = f(0,0)/f 1 (0) =.05/.4 =.125. Here f 1 (0) = f(, 0 y ) = f(0,0) + f(0,1) =.05 +.35 y =.4. 11
Contnuous Jont Dstrbutons The random varables x and y have a contnuous jont dstrbuton b t f there exsts a nonnegatve functon f defned on R 2 such that for any A R 2 Prob((x,y) A) = f( x, y) dxdy. A f s called the jont probablty densty functon of x and y. Propertes of f f satsfes the usual propertes: P.1 f(x,y) 0. P.2 f(x,y)dxdy = 1. 12
Dstrbuton functon F(x',y') = Prob(x x' and y y') = y' x' f(x,y)dxdy. If F s twce dfferentable, then w e have that f(x,y) = 2 F(x,y)/ x y. Margnal Densty and Dstrbuton Functons The margnal densty and dstrbuton functons are defned d as follows: a. F 1 (x) = lm y F(x,y) and F 2 (y) = lm x F(x,y). (margnal dstrbuton functons) b. f 1 (x) = fxy (,) dy and f 2 (y) = fxy (,) dx. y x 13
Example Let f(x,y) = 4xy for x,y [0,1] and 0 otherwse. a. Check to see that 1 0 1 4xydxdy = 1. 0 b. Fnd F(x',y'). Clearly, F(x',y') = 4 f(x,y). c. Fnd F 1 (x) and F 2 (y). We have that F 1 1( (x) = lm x 2 y 2 = x 2. y 1 Usng smlar reasonng, F 2 (y) = y 2. d. Fnd f 1 (x) and f 2 (y). y' 0 x' xydxdy = (x') 2 (y') 2. Note also that 2 F/ x y = 4xy = 0 1 f 1 (x) = f(x,y)dy = 2x and f 2 (y) = f(x,y)dx = 2y. 0 0 1 We have Condtonal Densty The condtonal densty functon of x, gven that y s fxed at a partcular value s gven by g 1 (x y) = f(x,y)/f 2 (y). Lkewse, for y we have g 2 (y x) = f(x,y)/f 1 (x). It s clear that g 1 (x y)dx = 1. 14
Condtonal Dstrbuton Functons We have The condtonal dstrbuton functons are gven by G 1 (x' y) = G 2 (y' x) = x' y' g 1 (x y)dx, g 2 (y x)dy. Example Let us revst example #2 above. We have that f = 4xy wth x,y (0,1). Moreover, g 1 (x y) = 4xy/2y = 2x and g 2 (y x) = 4xy/2x = 2y. x' G 1 (x' y) = 2 0 x dx = 2 ( x') 2 2 = (x') 2. By symmetry. G 2 (y' x) = (y') 2. It turns out that n ths example, x and y are ndependent random varables, because the condtonal dstrbutons do not depend on the other random varable. 15
Independent Random Varables Def. The random varables (x 1,...,x n ) are sad to be ndependent f for any n sets of real numbers A, we have Prob(x 1 A 1 & x 2 A 2 &...& x n A n ) = Prob(x 1 A 1 )Prob(x 2 A 2 ) Prob(x n A n ). Results on Independence The random varables x and y are ndependent d ff F(x,y) = F 1 (x)f 2 (y) or f(x,y) = f 1 (x)f 2 (y). Further, ff x and y are ndependent, then g 1 (x y) = f(x,y)/f 2 (y) = f 1 (x)f 2 (y)/ f 2 (y) = f 1 (x). 16
Extensons The noton of a jont dstrbuton can be extended d to any number of random varables. The margnal and condtonal dstrbutons are easly extended to ths case. Let f(x 1,...,x n ) represent the jont densty. Extensons The margnal densty for the th varable s gven by f (x ) =... f(x 1,...,x n )dx 1...dx -1 dx +1...dx n. The condtonal densty for say x 1 gven x 2,...,x n s g 1 (x 1 x 2,...,x n ) = f(x 1,...,x n )/ f(x 1,...,x n )dx 1. 17
Summary Measures of Probablty Dstrbutons Summary measures are scalars that convey some aspect of the dstrbuton. Because each s a scalar, all of the nformaton about the dstrbuton cannot be captured. In some cases t s of nterest to know multple summary measures of the same dstrbuton. There are two general types of measures. a. Measures of central tendency: Expectaton, t medan and mode b. measures of dsperson: Varance Expectaton The expectaton of a random varable x s gven by E(x) = x f(x ) (dscrete) E(x) = xf(x)dx. (contnuous) 18
Examples #1. A lottery. A church holds a lottery by sellng 1000 tckets at a dollar each. One wnner wns $750. You buy one tcket. What s your expected return? E(x) =.001(749) +.999(-1) =.749 -.999 = -.25. The nterpretaton s that f you were to repeat ths game nfntely your long run return would be -.25. #2. You purchase 100 shares of a stock and sell them one year later. The net gan s x. The dstrbuton s gven by. (-500,.03), (-250,.07), (0,.1), (250,.25),(500,.35), (750,.15), and (1000,.05). E(x) = $367.50 Examples #3. Let f(x) = 2x for x (0,1) and = 0, otherwse. Fnd E(x). 1 E(x) = 2x 2 dx = 2/3. 0 19
Propertes of E(x) P.1 Let g(x) be a functon of x. Then E(g(x)) s gven by E(g(x)) = g(x ) f(x ) (dscrete) E(g(x)) = g(x)f(x) dx. (contnuous) P.2 If k s a constant, then E(k) = k. P.3 Let a and b be two arbtrary constants. Then E(ax + b) = ae(x) + b. Propertes of E(x) P.4 Let x 1,...,x n be n random varables. Then E( x ) = E(x ( ). P.5 If there exsts a constant k such that Prob(x k) = 1, then E(x) k. If there exsts a constant k such that Prob(x k) = 1, then E(x) k. P.6 Let x 1,...,x n be n ndependent random varables. Then E( x ) = Ex ( ). n 1 n 1 20
Medan Def. If Prob(x m).5 and Prob(x m).5, then m s called a medan. a. The contnuous case m fxdx () = f() x dx =.5. m b. In the dscrete case, m need not be unque. Example: (x 1,f(x 1 )) gven by (6,.1), (8,.4), (10,.3), (15,.1), (25,.05), (50,.05). In ths case, m = 8 or 10. Mode Def. The mode s gven by m o = argmax f(x). A mode s a maxmzer of the densty functon. It need not be unque. 21
A Summary Measure of Dsperson: The Varance In many cases the mean the mode or the medan are not nformatve. In partcular, two dstrbutons wth the same mean can be very dfferent dstrbutons. One would lke to know how common or typcal s the mean. The varance measures ths noton by takng the expectaton of the squared devaton about the mean. Varance Def. For a random varable x, the varance s gven by E[(x- ) 2 ], where = E(x). The varance s also wrtten as Var(x) or as 2. The square root of the varance s called the standard devaton of the dstrbuton. It s wrtten as. 22
Illustraton Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec E(x) Var(x) Lawrence 28 34 46 57 66 75 80 78 70 58 46 34 56 334 Santa Barbara 52 54 55 57 58 62 65 67 66 62 57 52 58.91667 28.62879 Computaton: Examples a. For the dscrete case, Var(x) = (x - ) 2 f(x ). As an example, f (x, f(x )) are gven by (0,.1), (500,.8), and (1000,.1). We have that E(x) = 500. Var(x) = (0-500) 2 (.1) + (500-500) 2 (.8) + (1000-500) 2 (.1) = 50,000. b. For the contnuous case, Var(x) = (x- ) 2 f(x)dx. Consder the example above where f = 2x wth x (0,1). From above, E(x) = 2/3. Thus, 1 Var(x) = (x - 2/3) 2 2x dx = 1/18. 0 23
Propertes of Varance P.1 Var(x) = 0 ff there exsts a c such that Prob(x = c) = 1. P.2 For any constants a and b, Var(ax +b) = a 2 V ar(x). P.3 Var(x) = E(x 2 ) - [E(x)] 2. P4 P.4 Ifx, = 1,...,n, are ndependent, then Var( x )= Var(x ). P.5 If x are ndependent, = 1,...,n, then Var( a x ) = a 2 Var(x ). A remark on moments Var (x) s sometmes called the second moment about the mean, wth E(x- ) = 0 beng the frst moment about the mean. Usng ths termnology, E(x- ) 3 s the thrd moment about the mean. It can gve us nformaton about the skewedness of the dstrbuton. E(x- ) 4 s the fourth moment about the mean and t can yeld nformaton about the modes of the dstrbuton or the peaks (kurtoss). 24
Moments of Condtonal and Jont Dstrbutons Gven a jont probablty densty functon f(x 1,..., x n ), the expectaton of a functon of the n varables say g(x 1,..., x n ) s defned as E(g(x 1,..., x n )) = g(x 1,..., x n ) f(x 1,..., x n ) dx 1 dx n. If the random varables are dscrete, then we would let x = (x )bethe th 1,..., x n observaton and wrte E(g(x 1,..., x n )) = g(x ) f(x ). Uncondtonal expectaton of a jont dstrbuton Gven a jont densty f(x,y), E(x) s gven by E(x) = xf 1 (x)dx = Lkewse, E(y) s xf(x,y)dxdy. E(y) = yf 2 (y)dy = yf(x,y)dxdy. 25
Condtonal Expectaton The condtonal expectaton of x gven that x and y are jontly dstrbuted b t d as f(x,y) s defned by (I wll gve defntons for the contnuous case only. For the dscrete case, replace ntegrals wth summatons) E(x y) = xg 1 (x y) dx Condtonal Expectaton Further the condtonal expectaton of y gven x s defned d analogously l as E(y x) = yg 2 (y x) dy 26
Condtonal Expectaton Note that E(E(x y)) = E(x). To see ths, compute E(E(x y)) = [ xg 1 (x y)dx]f 2 dy = { x[f(x,y)/( f 2 )]dx}f 2 dy = x f(x,y)dxd dy, and the result holds. Covarance. Covarance s a moment reflectng drecton of movement of two varables. It s defned as Cov(x,y) = E[(x- x )(y- y )]. When ths s large and postve, then x and y tend to be both much above or both much below ther respectve means at the same tme. Conversely when t s negatve. 27
Computaton of Cov Computaton of the covarance. Frst compute (x- x )(y- y ) = xy - x y - y x + x y. Takng E, E(xy) - x y - x y + x y = E(xy) - x y. Thus, Cov(x, y) = E(xy) - E(x)E(y). If x and y are ndependent, then E(xy) = E(x)E(y) and Cov(xy) = 0. 28