Point Estimation-III: General Methods for Obtaining Estimators

Transcription

1 Pot Estmato-III: Geeral Methods for Obtag Estmators RECAP Data: Radom Sample from a Populato of terest o Real valued measuremets: o Assumpto (Hopefully Reasoable) o Model: Specfed Probablty Dstrbuto f ( x ) o Ivolves some ukow parameter(s). o Wsh to lear about from the Data (Estmato) May estmators for parameter or some fucto u( ) o Some may use all the data, e.g., Sample Based: Mea, Meda, Varace o Some may gore data! Bad, ad should be avoded. Desrable propertes of estmators o Cosstet o Ubased Cramer-Rao Iequalty Mmum Varace Boud Fsher Iformato o Statstc (some fucto of the data) - Summary May ot cota all relevat fo. the data about

2 o Statstc(s) that cota all the formato about are called Suffcet Statstc(s) No Iformato lost f oly ths statstc s stored Codtoal Dstrbuto of Data gve the statstc(s) does ot deped o the ukow Factorzato Theorem: yelds suffcet statstcs, e.g., o B. Beroull ( ) o B.7 Posso ( ) - X, X X X, X X o C.7 Normal (Ukow mea ) X or X X o C.7 Normal (Ukow mea ad varace) - X X X X X, ( ),or,, or X, ( X X), X, ( X X) o C.4 Expoetal ( ) X, X X o C.9 Uform(0,) X ( ) max ( X ),,, o C.9 Uform X () m ( X), X( ) max ( X) o C.6 Gamma(,,,,,, X, X, or X, X, or l X, X

3 Several popular methods yeld o Estmators wth desrable propertes Study two methods o Method of Momets (MOM) Secto 0.7 o Maxmum Lkelhood Estmato (MLE)- Secto 0.8 Secto Method of Momets (MOM) k th momet of a dstrbuto o Populato Momet: th k Raw Momet: For k =,, k k Dscrete dstrbuto: EX [ ] x f x k k k Cotuous Dstrbuto: EX [ ] x f x dx. k These momets deped o the parameter(s) fuctos of the varables. Note: EX [ ] ; EX [ ] VarX ( ) (**) o Sample Momets: Gve radom sample X,, X, act as f the kow dscrete dstrbuto f ( x ),,,, s a approxmato of the populato dstrbuto That s what Mote Carlo Smulato does! th k k Sample Raw Momet: For k =,, m k x.

4 Smple Idea- MOM: Equate frst few populato momets to the correspodg sample momets ad solve for the ukow parameters. o For oe parameter problems, set m X. Examples: Check oe parameter dstrbutos lst For may of the dstrbutos lsted above, the ukow parameter EX ( ), therefore ˆ X. For some others, t s ot. Whch oes! MOM o For two parameter problems, use two equatos wth two ukows: Set X ad X. Usg (**), soluto of these equatos leads to ˆ MOM X X X (***) ad ˆ MOM ( ). Gve a specfc dstrbuto, oe obtas the populato mea ad varace as a fucto of the ukow parameters ad sets them equal to ˆ ad ˆ MOM MOM ad solve. E.g., Normal (, ). Gamma Uform

5 Note: MOM estmators may or may ot be o - fucto of Suffcet Statstc(s) Check whch oes do. o Ubased Check whch are. Secto 0.8 Maxmum Lkelhood Estmators (MLE) Motvato: Gve the observed sample, pck the value of the parameter that would seem to maxmze the observed values probablty uder the model. Cauto: Do t forget that after the data has bee observed, there s o radomess those observatos. The jot dstrbuto f( x,, x; ) f( x ) of the radom varables X,, X chages wth the values of the ukow parameter Gve the observed sample x,, x, treat L( ) f( x,, x ; ) as a fucto of the parameter as a varable. The fucto L( ) s called the lkelhood fucto. Examples: Posso, Bomal, Expoetal, Normal

6 Key dea ML METHOD: MLE of the parameter ˆ MLE :. The value amog all possble values of the parameter that maxmzes the lkelhood fucto.. Maxmze the lkelhood L( ) wth respect to. The resultg value at whch the maxmum s acheved s called the MLE ˆ of. MLE. EXAMPLE : Two supplers of a electroc compoet. Possble values of the percetage of defectves are: Suppler : 5% ad Suppler : 0%. [Suppler costs less.] We do t kow whether a large cosgmet the warehouse came from Suppler or Suppler. So we take a radom sample of 00 uts from ths cosgmet ad measure the proporto of defectves the sample. Assume that the dstrbuto of the # of defectves the sample s bomal b(00, p) where 00*p s oe of the values above. How does oe fd the MLE gve that the umber of defectves the sample s x?

7 v. Examples Oe parameter problems: Posso, Uform, Expoetal, Normal

8

9 RECAP: Rgorous steps eeded to check that the maxma s acheved. d d. Set the frst dervatve ( ) log e L( ) 0ad fd d d the soluto ths equato.. Make sure that the secod dervatve ˆ s a relatve maxma of ( ). d d ˆ ( ) 0,.e.. But we wll ot worry about the secod step our applcatos. Problems wth two parameters: Normal, Uform, Gamma

10

11

12

13

14 RECAP: Steps eeded to check that the maxma of the loglkelhood fucto, (, ) a fucto of two varables, s acheved.. Set the frst partal dervatves (, ) 0,,. ad ˆ ( ˆ, ˆ ) fd the soluto these equatos.. Make sure that at least oe of the secod partal. dervatves, The Hessa ˆ (, ),, s egatve. (, ) (, ) (, ) (, ) (, ) (, ) (, ) ˆ ( ˆ, ˆ ) evaluated at s postve. v..e., ˆ s a relatve maxma of (, ). Aga, we wll ot eforce ths level of rgor 4. Gve the data, oe ca always fd the ML estmate by usg umercal optmzato tools.