Today s outline: pp

Transcription

1 Chapter 3 sections We will SKIP a number of sections Random variables and discrete distributions Continuous distributions The cumulative distribution function Bivariate distributions Marginal distributions Conditional distributions SKIP Multivariate distributions (generalization of bivariate) Functions of a random variable Functions of two or more random variables SKIP Markov Chains

2 Today s outline: pp Cumulative distribution function Multiple random variables 1. Joint probabilities 2. Marginal distributions 3. Conditional distributions 4. Independence

3 Definition: Cumulative distribution function (cdf) The cdf of a random variable X, denoted by F (x) is defined by F (x) = P(X x) = x f (y)dy, for all x. In practice, we use the pdf much more frequently than the cdf. However, the cdf has some additional theoretical properties (e.g. uniqueness) that the pdf doesn t have. Relationship between cdf and pdf F (x) = x f (y)dy f (x) = d dx F (x)

4 Example : Tossing 3 coins F (x) = 0 < x < 0 1/8 0 x < 1 1/2 1 x < 2 7/8 2 x < x < For example, F (2.5) = P(X 2) = P(X = 0, 1, 2) = 7/8. Cdf can be discontinuous, but F (x) take values at top of jumps.

5 Theorem The function F (x) is a cdf if and only if the following three conditions hold: a. lim x F (x) = 0 and lim x F (x) = 1. b. F (x) is a nondecreasing function of x. c. F (x) is right-continuous; that is, for every number x 0, lim x x0 F (x) = F (x 0 ).

6 Example: geometric distribution (discrete) Let p be the probability of tossing a head, X the number of tosses until the first head comes up.

7 Example: geometric distribution (discrete) Let p be the probability of tossing a head, X the number of tosses until the first head comes up. which gives Verify that F (x) is a cdf. P(X = x) = (1 p) x 1 p, F (x) = P(X x) = 1 (1 p) x.

8 Definition A random variable is continuous is F (x) is continuous, and discrete if Fx is a step function. Example: continuous cdf Special case of the logistic function: F (x) = e x.

9 Uniqueness Assuming certain non-pathological conditions, cdf completely determines the probability distribution of a random variable. Definition The random variables X and Y are identically distributed if, for every set A, P(X A) = P(X B). Note: X and Y are not necessarily the same, take for example heads and tails of a fair coin. Both have the same distribution, but they are not the same thing. Theorem The following statements are equivalent: a. The random variables X and Y are identically distributed. b. F X (x) = F Y (x) for every x.

10 Discrete bivariate random variables 1. Discrete case X, Y, joint probability function: P(X = x, Y = y), (pmf ). Example Randomly draw from {1, 2, 3, 4, 5} X =1st draw Y =2nd draw without replacement What is P(X = 4, Y = 5)? Definition Let (X, Y ) be a discrete bivariate random vector. Then f (x, y) = P(X = x, Y = y) (or f X,Y (x, y)) is called the joint probability mass function.

11 2. Discrete case X, Y, joint cumulative distribution function: F (x, y) = P(X x, Y y)

12 3. Marginal distribution: Theorem Let (X, Y ) be a discrete bivariate random vector with joint pmf f X,Y (x, y). Then the marginal pmfs of X and Y, f X (x) = P(X = x) and f Y (y) = P(Y = y), are given by f X (x) = y R f X,Y (x, y) and f Y (y) = x R f X,Y (x, y). 4. The conditional distribution of a discrete random variable Y on a r.v. X with joint pmf f X,Y (x, y) and marginals f X (x) and f Y (y) is f Y (y X = x) = P(Y = y X = x) = f X,Y (x, y). f X (x) Similarly, the conditional expectation of Y given X is E (g(x, Y ) X = x) = y R g(x, y)p(y = y X = x).

13 5. Two random variables X and Y with joint pmf f X,Y (x, y) and marginal pmfs f X (x) and f Y (y) are independent if FOR ALL x R and y R: f X,Y (x, y) = f X (x)f Y (y). Compare it with independent events. Example: 2x2 tables

14 Continuous variables 1. A function f (x, y) is a joint probability density function (pdf) of the continuous bivariate r.v. (X, Y ) if FOR ALL A R 2 : P((X, Y ) A) = f (x, y)dxdy. The function f (x, y) is a surface over XY plane, such that: f (x, y) 0 (x, y) R 2. f (x, y)dxdy = Joint cumulative distribution function (cdf) F (x, y) = P(X x, Y y) = Relationship between cdf and pdf 2 F (x, y) x y A x = f (x, y). y f (s, t)dtds.

15 Continuous joint pdf examples f (x, y) = 4xy 0 < x < 1, 0 < y < 1. What is P(X < Y )? What is P(Y > X 2 )? What is P(X + Y 0.5)?

16 Continuous joint pdf examples (ctd) Bivariate normal pdf

17 3. The marginal pdfs are defined as in the discrete case, with integrals instead of sums (as in the discrete case) f X (x) = f Y (y) = f (x, y)dy, < x < f (x, y)dx, < y <

18 Properties F X,Y (x, y) is non-decreasing in x, y. F X,Y (x, y) is right-continuous in x, y. and lim x F X,Y (x, y) = 0, lim F X,Y (x, y) = 0, y lim F X,Y (x, y) = 1. y,x FOR ALL x 1 x 2 and y 1 y 2, F (x 2, y 2 ) F (x 2, y 1 ) F (x 1, y 2 ) + F (x 1, y 1 ) 0. Draw picture!

19 4. The conditional distribution of Y given X = x is given by f Y (y X = x) = f XY (x, y). f X (x) Note that here we are conditioning on something which has probability 0! Check that this is indeed a pdf. Example: joint Gaussian 5. Two continuous random variables X and Y with joint pdf f X,Y (x, y) and marginals f X (x) and f Y (y) are independent if FOR ALL x and y, f X,Y (x, y) = f X (x)f Y (y).