Probability Model for 2 RV s

Transcription

1

2

3 Probability Model for 2 RV s The joint probability mass function of X and Y is P X,Y (x, y) = P [X = x, Y= y] Joint PMF is a rule that for any x and y, gives the probability that X = x and Y= y. 3

4 Example: Joint PMF Test n = 2 circuits Assume outcomes are independent. Probability of accept or P [a] = 0.9. X = number of acceptable circuits. Y = number of tests before the first rejection. 4

5 Example: Joint PMF P X Y (x, y) y = 0 y = 1 y = 2 x = x = x =

6 Experiment produces X and Y. All you care about is X. Marginal PMF PMF of X is P X (x) =. P X,Y (x, y) y S Y Calculate by summing across rows, write PMF in margin 6

7 Example: PMFs from Joint PMF Experiment produces X and Y. We care about either X or Y. PMFs of X and Y: 7

8 Derived rv W = g(x, Y ) Functions of 2 RVs PMF of W: P W (w) =. (x,y):g (x,y)=w P X,Y (x, y) 8

9 Derived PMF Example Example W = XY: 9

10 Expectations of Functions again Thm: The expected value of W = g(x,y ) is E[W ] =.. g(x, y)p X,Y (x, y) x S X y S Y If g(x, Y ) = g 1 (X, Y ) + g 2 (X, Y ) + + g k (X, Y ), then E[g(X, Y )] = E[g 1 (X, Y )] + + E[g n (X, Y )] 10

11 Expectations of Sums E[X + Y ] = E[X] + E[Y ] With W = ( X + Y µ X + Y ) 2, E[W ] = Var [X + Y ] = Var [X] + Var [Y ] + 2E[(X µ X )(Y µ Y )] 11

12 Covariance Covariance: W = ( X µ X )(Y µ Y ), Cov [X, Y ] = E[W ] = E[(X µ X ) (Y µ Y )] Covariance is also Cov [X, Y ] = E[XY ] µ X µ Y Cov [X, Y ] > 0 says, X > E[X] implies Y> E[Y ] is likely (X goes up, Y goes up) Var [X + Y ] = Var [X] + Var [Y ] + 2 Cov [X, Y ] 12

13 Correlation The correlation of X and Y is E[XY ] Correlation = Covariance if E[X] = E[Y ] = 0 E[XY ] > 0 suggests that X > 0 increases chance Y > 0 Orthogonal: E[XY ] = 0 Uncorrelated: Cov [X, Y ] = 0 13

14 Correlation Coefficient The correlation coefficient of two random variables X and Y is ρ X,Y Cov [X, Y] =, Var [X] Var [Y ] Thm: the Correlation coefficient is normalized: 1 ρ X,Y 1 Thm: If Y= ax + b, then 1 a < 0 ρ X,Y = 0 a = 0 1 a > 0 14

15 Independent random variables X and Y are independent if and only if {X = x} and {Y= y} are independent events for all x and y Equivalently, X and Y are independent if P X,Y (x, y) = P X (x) P Y (y) 15

16 Properties of Independent RVs For independent random variables X and Y, r X,Y = E[XY ] = E[X]E[Y ] E[X Y = y] = E[X] for all y S Y E[Y X = x] = E[Y ] for all x S X Var [X + Y ] = Var [X] + Var [Y ] Cov [X, Y ] = ρ X,Y = 0 16

17 Multiple Discrete RVs The joint PMF of the discrete random variables X 1,..., X n is P X 1,...,X n (x 1,..., x n ) = P[X 1 = x 1,..., X n = x n ] 17

18 Multiple Continuous Random Variables (RVs) Experiment produces at least two continuous RVs The joint CDF of X and Y is F X, Y (x, y) = P [X x, Y y] 18

19 Joint CDF Area Y {X<x, Y<y} (x,y) X 19

20 Joint CDF Properties 0 F X, Y (x, y) 1 F X (x) = P [X x] = P [X x, Y < ] = F X, Y (x, ) F Y (y) = F X, Y (, y) F X, Y (, y) = F X, Y (x, ) = 0 F X, Y (, ) = 1 If x 1 x and y 1 y, then F X, Y (x 1, y 1 ) F X, Y (x, y) 20

21 Joint PDF The joint PDF of X and Y is f X, Y (x, y) satisfying x y F X, Y (x, y) = f X, Y (u, v) dv du Above definition implies f X, Y (x, y) = 2 F X,Y (x, y) x y 21

22 Joint PDF (continued) f X, Y (x, y) measures probability per unitarea Probability that X, Y is in the set {x < X x + dx, y < Y y + dy} is P[x < X x + dx, y < Y y + dy] = f X, Y (x, y) dx dy 22

23 f X, Y (x, y) 0 for all (x, y) f X, Y (x, y) dx dy = 1 Joint PDF Properties Probability of an event A that corresponds to a region in the X,Y plane is P [A] = A f X, Y (x, y) dxdy 23

24 Joint PDF: Example Given: RVs X and Y have joint PDF f X, Y (x, y) = c 0 0 x 3, 0 y 5 otherwise Find: c =? If A = {Y> X}, then P [A] =? 24

25 Comments Obtaining joint CDF, F X, Y (x,y) from joint PDF, f X, Y (x,y) could be TRICKY!!!; see Example

26 Marginal PDF Experiment produces two continuous RVs X and Y with joint PDF f X, Y (x, y), Marginal PDFs are obtained from joint PDF as follows f X (x) = f X, Y (x, y) dy f Y (y) = f X, Y (x, y) dx 26

27 Marginal PDF: Example Given: RVs X and Y have joint PDF f X, Y (x, y) = 2x 0 x 1, y x 2 0 otherwise Find: f X (x) =? f Y (y) =? 27

28 Functions of Two Random Variables Example: Receiver outputs X and Y from two antennas W 1 = max(x, Y ) W 2 = X + Y W 3 = ax + by PDF of W i =? Find the CDF of W i first 28

29 Example Given: RVs X and Y have joint PDF f X, Y (x, y) = W = max(x, Y ) Find: f W (w) =? 1/ x 3, 0 y 5 otherwise 29

30 Expected Values W = g(x, Y). Method 1: Find the PDF f W (w), calculate E[W ] = wf W (w) dw Method 2: E[W ] = E[g(X, Y)] = g(x, y )f X, Y (x, y) dx dy 30

31 Expectation of Sums The expected value of g(x, Y ) = g 1 (X, Y ) + + g n (X, Y ) is E[g(X, Y )] = E[g 1 (X, Y )] + + E[g n (X, Y )] Sums: E[X + Y] = E[X] + E[Y ] Var [X + Y ] = Var [X] + Var [Y ] + 2 Cov [X, Y ] Covariance: Cov [X, Y ] = E[(X µ X ) (Y µ Y )] = E[XY ] µ X µ Y 31

32 Correlation Coefficient The correlation coefficient of X and Y is ρ X,Y Cov [X, Y] =, V a r [X] Var [Y ] Theorem: 1 ρ X,Y 1 Proof is the same as for discrete random variables 32

33 Independent Continuous RVs Continuous RVs X and Y are independent iff f X, Y (x, y) = f X (x) f Y (y) for all x andy Properties of Independent RVs: E[g(X)h(Y )] = E[g(X)]E[h(Y )] Cov [X, Y ] = 0 Var [X + Y ] = Var [X] + Var [Y ] 33

34 Are X and Y independent? Independence: Example 1 f X, Y (x, y) = 4xy 0 x 1, 0 y 1 0 otherwise The marginal PDFs of X and Y are f X (x) = f Y (y) = 2x 0 x 1 0 otherwise 2y 0 x 1 0 otherwise f X, Y (x, y) = f X (x) f Y (y) = X and Y are independent! 34

35 Are U and V independent, when Independence: Example 2 f U,V (u, v) = 24uv 0 u 0, v 0, u + v 1 otherwise Region of nonzero density is triangular and f U (u) = f V (v) = 12u(1 u) v(1 v) u 1 otherwise 0 v 1 otherwise f U,V (u, v) ƒ= f U (u) f V (v) = U and V are not independent! 35

36 Jointly Gaussian Random Variables X and Y have a bivariate Gaussian PDF if. x µ1. 2 2ρ(x µ 1 )(y µ 2 ). y µ2. 2 f X, Y (x, y) = exp σ 1 σ 1 σ 2 + σ 2 2(1 ρ 2 ) 2πσ 1 σ 2, 1 ρ 2 σ 1 > 0, σ 2 > 0, 1 ρ 1 36

37 ρ = 0 f X, Y (x, y) x y µ 1 = µ 2 = 0 σ 1 = σ 2 = 1 37

38 ρ = 0.9 f X, Y (x, y) x y µ 1 = µ 2 = 0 σ 1 = σ 2 = 1 38

39 ρ = 0.9 f X, Y (x, y) x y µ 1 = µ 2 = 0 σ 1 = σ 2 = 1 39

40 Rewriting the Bivariate Gaussian PDF Manipulating the bivariate Gaussian PDF, we obtain where 2 2 f X, Y (x, y) = 1 e (x µ 1 ) /2σ 1 1 σ1 2π σ 2 2π µ 2(x) = µ 2 + ρ σ 2 σ1 σ 2(x) = σ 2, 1 ρ 2 (x µ 1) 2 2 e (y µ 2 ( x )) /2σ 2 40

41 Theorem 5.18: Theorem from the Bivariate Gaussian PDF X N [µ, σ ] Y N [µ, σ ] 41

42 More than Two Continuous RVs The joint CDF of X 1,..., X n is F X 1,...,X n (x 1,..., x n ) = P[X 1 x 1,..., X n x n ] The joint PDF of X 1,..., X n is f X 1,...,X n (x 1,..., x n ) satisfying F X 1,...,X n (x 1,..., x n ) x 1 x n = f X 1,...,X n (u 1,..., u n ) du 1 du n 42

43 Joint PDF Properties f X 1,...,X n (x 1,..., x n ) = n F X 1,...,Xn (x 1,...,x n ) x 1 x n f X 1,...,X n (x 1,..., x n ) 0 f X 1,...,X (x n 1,..., xn 1 dx= n 1 ) dx P[A] = f X 1,...,X n (x 1,..., x n ) dx 1 dx 2... dx n A 43

44 Marginal PDFs For a joint PDF of four RVs, f W, X, Y, Z (w, x, y, z), some marginal PDFs are f X, Y, Z (x, y, z) = f Y, Z (y, z) = f Z (z) = f W, X, Y, Z (w, x, y, z) dw f W, X, Y, Z (w, x, y, z) dw dx f W, X, Y, Z (w, x, y, z) dw dx dy 44

45 N Independent Random Variables X 1,..., X n are independent if f X 1,...,X n (x 1,..., x n ) = f X 1 (x 1 ) f X n (x n ) for all x 1,..., x n. X 1,..., X n are independent and identically distributed or iid if f X 1,...,X n (x 1,..., x n ) = f X (x 1 ) f X (x n ) for all x 1,..., x n. 45

46 Expectations For Y= g(x 1,..., X n ), E[Y ] = E[g(X 1... X n )] = g(x 1... x n ) f X 1...X n (x 1... x n ) dx 1 dx n 46

47 Problem Let X 1,..., X n be iid rv s, mean 0, var 1, covariance Cov [X i, X j ] = ρ. Find the expected value and variance of the sum Y = X X n. 47

48 Problem Let X 1,..., X n be iid random variables, each with PDF f X (x). Find the CDF and PDF of Y= min {X 1,..., X n }. 48