SCOREM 2.11: A PROGRAM FOR THE ESTIMATION OF GENERAL STATE-SPACE MODELS WITH THE EM AND SCORING ALGORITHMS

Transcription

1 SCOREM 2.: A PROGRAM FOR THE ESTIMATION OF GENERAL STATE-SPACE MODELS WITH THE EM AND SCORING ALGORITHMS Jacques Raynauld * Jean-Guy Simonao Chrisian Sigouin Insiu d'économie appliquée École des Haues Éudes Commerciales 5255 Decelles, Monréal, H3T V6 Phone: Fax: Inerne: RAYNAULD@HEC.CA Augus 993 * We would like o acknowledge he financial suppor of he Social Sciences and Humaniies Research Council of Canada, Les Fonds FCAR of he Québec governmen, Les Fonds Faribeaul of he Universié de Monréal and he Cener for Inernaional Business Sudies of l'école des Haues Éudes Commerciales. Correspondence should be addressed o he firs auhor.

2 . INTRODUCTION This paper describes how o use a program available on a companion diskee o esimae dynamic models wih unobservable variables (also called DYMIMIC models). This program works wih GAUSS 2. and 3.. The user is expeced o have some basic noions concerning his programming language. We firs begin by explaining he esimaion mehods used by he program. We hen show how o se up a model by providing a deailed example. The auhors welcome any commens concerning possible improvemens or problems wih his version of SCOREM. 2. ESTIMATION METHODS This program is designed o esimae DYMIMIC models using eiher a Newon scoring algorihm or an expecaion-maximizaion (EM) algorihm. These are wo maximum likelihood based esimaion mehods which rely on he Kalman filer. Boh have relaive srenghs and weaknesses. The scoring algorihm [described in Engle and Wason (98)] is fairly accurae and converges relaively rapidly bu is highly dependen on he iniial values given o he model parameers. Inappropriae iniial values lead o many problems such as non-posiive definie variance-covariance marices which make he esimaion process very difficul if no impossible. In many cases, finding a good se of iniial values could be difficul and an exensive search can be exremely ime consuming. Unlike he scoring algorihm, he EM algorihm [described in Wason and Engle (983), Shumway and Soffer (982), and Shumway (988)] is no very sensiive o iniial values since i uses OLS o find he parameer vecor maximizing he likelihood. A few ieraions wih his algorihm will do a good job finding a parameer vecor close o he one maximizing he likelihood funcion. However, his algorihm is slow o converge and unlike he scoring algorihm, i doesn' provide any sandard errors for he esimaed parameers. This program is buil in a manner ha permis he sequenial use of hese wo algorihms. This way, we can ake advanage of boh mehods' srenghs and overcome Following Ruud (99), i is possible o consruc an esimae of he Hessian using he score. However, his possibiliy is no available in he curren version of he program.

3 mos of heir weaknesses. A few ieraions wih he EM algorihm can be used o find a "nearly opimal" parameers vecor which will no cause any problem if used as iniial values for he scoring algorihm. The laer can hen be used o find he final esimaes and heir associaed sandard errors. However, wih he new "squeezing" feaure conrolling he sep-lengh parameer added o SCOREM 2., he user can mos of he ime bypass he EM algorihm and sar righ away wih he scoring algorihm. 2 The possibiliy o use analyical derivaives for he scoring algorihm wihou much work is also a program's main feaure. This permis o speed up he esimaion process from wo o hree imes compared o a sandard numerical derivaives based mehod. 3. INSTALLATION The program should be insalled as i is on he diskee, ha is, wih a main direcory named SCOREM (or any oher name) and wo sub-direcories named EM and SCORING. Examples can be found in he sub-direcory EXAMPLES. The resuls from each esimaion will also be sored in he main direcory. 4. SETTING UP A STATE-SPACE MODEL The program is designed o work wih he sae-space formulaion of a model [see Harvey (98, 989) for furher explanaions] ha is, a measuremen equaion which links he observed variables (Y) o he sae variables (α ): Y = Z α + β X + S e () (nx) (nxm) (mx) (nxk) (kx) (nxnm) (nmx) and a ransiion equaion which describes he process followed by he sae variables: α = T α -+ γ X 2 + R n (2) 2 (mx) (mxm) (mx) (mxk2) (k2x) (mxg) (gx) e _ N(, H ) and n _ N(, Q ) 3 where e and n are random disurbances assumed o follow a normal disribuion wih variance-covariance marices H and Q respecively. The dimensions of each marix are in parenhesis. X and X2 are vecors of exogenous variables. A paricular model is se up by providing he conens of each marix in equaions

4 () and (2). Acually, his ask consiss in discriminaing beween fixed and free parameers (he ones which are o be esimaed). Of course, his should always be done in a manner so as o avoid idenificaion problems. Underidenified models will invariably lead o a program failure. A simple rule of humb is o fix a leas one parameer for each sae variable. Moreover, pracice has shown ha i is a good idea o sandardize all observable variables (The Ys and he Xs in he measuremen equaion), a leas in he saionary case. This helps he algorihms o converge by puing all variables on he same scale. Furhermore, if for idenificaion purposes a variance is fixed, is value should be in line wih he size of he corresponding variables: his will preven he oher esimaed variances from becoming negaive. Furher idenificaion ips are given in he deailed examples of he appendices. 3 Each differen model mus be iniialized wih he <name>.gau and <name>.prc files. A simple example illusraes how his can be done. 5. SETTING UP A MODEL: AN EXAMPLE Suppose we wan o esimae a coinciden indicaor of economic aciviy in a given counry (see Sock and Wason, 99). I seems reasonable o suppose ha variables like employmen (EMP), manufacurers' shipmens (SHIP) and reail sales (RTS) would be driven by a common (unobservable) variable named economic aciviy (EA) and by some idiosyncraic componens (U, U2, and U3). As explained before, e, e2, e3 and n are normally disribued random disurbances. A firs difference was applied o all variables which are now saionary (he variables were also sandardized). The model o esimae would hence be: SHIP Z EMP = Z RTS Z 2 3 for he measuremen equaion and EA β U + β U 2 β U [] 4

5 4 EA T U T 22 = U 2 T 33 U 3 T 44 for he ransiion equaion, wih Q Q = 6. Q22 Q33 EA U U 2 U γ γ + γ γ [] + n e 5 e 2 e 3 We need o make hree remarks. Firs, our model doesn' conain any exogenous variable. However, we see ha he marices β and γ are sill par of i. This is because we filled he marices X and X2 wih zeros. I is he way o proceed: in any case, he dimensions of a marix mus never be less han one by one. Marices which are no used mus be filled wih zeros. Second, we mus address idenificaion issues. As saed before, here mus be a leas one ied parameer for each unobservable variable. We have four unobservable variables: we need o fix a leas four parameers. Obvious candidaes are he idiosyncraic componens' coefficiens in he measuremen equaion. We se hem o. This leaves us wih one more parameer o fix. We decided o se he variance of EA o one. Alernaively, we could have se one more parameer o one in marix Z (for example, Z which ies he unis of measuremen of he unobservable variable EA o he observable variable SHIP). Eiher ways, i leaves us wih parameers o esimae. The vecor of parameers o esimae (named DELTA in he <name>.gau file) will hen have en elemens. Third, we can see ha here is no measuremen errors (no random disurbances in he measuremen equaion). Normally, his paricular case canno be reaed by he EM algorihm because i uses OLS on he measuremen equaion o find he parameers' value. The program overcomes his problem by auomaically adding small random disurbances o he measuremen equaion when i is needed. This is one more reason o use he scoring algorihm o find he final esimaes.

6 Appendices and 2 show how o program his paricular saionary model. The code corresponding o his model can be found in he files DEMO.GAU and DEMO.PRC OUTPUT FILES The program will produce wo oupu files: One for he EM algorihm and one for he SCORING algorihm. We are mainly concerned wih he laer. Running he previous model will give he following resuls: ITERATION # COEFFICIENTS VECTOR ITERATION ITERATION GRADIENT T-TEST K K- K- K LIKELIHOOD ITERATION ITERATION DIFFERENCE K K The firs par gives he value for he parameers vecor a he K h ieraion, a he (K-) h ieraion and he sandard errors associaed wih each elemen. -saisics are also provided. The second par provides he value of he Gaussian log-likelihood funcion (excluding he consan) a he Kh ieraion. I is compued as

7 T T - L= - log F - v F v = = Esimaed unobservable variables, variance-covariance marices, residuals and heir auocorrelaion as well as smoohed unobservable variables and VCV marices are all available a he end of he oupu files ESTIMATING VARIANCES Esimaing variance erms can someimes be a bi difficul, paricularly in he nonsaionary case. If he real variance erm is acually very small, some problems may arise due o he compuer's precision. In his case, i can push he esimaed variance owards a negaive value. In some oher cases, he gradien associaed wih a variance erm may no end owards zero. A way o overcome hese problems is o esimae he sandard error insead of he variance. This is done by using he square of he parameer in he.prc file. In he firs example, we could have used Q[2,2]=DELTA[8,]^2 insead of Q[2,2]=DELTA[8,] o esimae he firs variance erm. 8. OTHER FEATURES. In he saionary case, he program can compue he iniial variance-covariance marix of he sae vecor according o he srucure of he model posulaed.. By choosing he appropriae opions, he program can also used o esimae non saionary models as illusraed in Appendix 3 [see Pary, Raynauld, and Simonao (989) and Slade (989) for furher illusraions].. In mos of he cases Z is no ime varying and conains coefficiens o be esimaed. SCOREM 2. can now handle ime-varying Z marix such as in ime-varying regression models, Sims' BVAR models, ec. This possibiliy only works wih he scoring algorihm (wih boh analyic and numerical derivaives).. The program can also handle linear resricions wihin and across equaions. This las possibiliy is no currenly documened.

8 7 REFERENCES Engle, R.F. and M. Wason (98) A one facor mulivariae ime series model of meropoliain wage raes, Journal of he American Saisical Associaion, 76, Harvey, A.C. (98) Time Series Mehods, Halsed Press, New-York. Harvey, A.C. (989) Forecasing Srucural Time Series Models and he Kalman Filer, Cambridge Universiy Press, Cambridge. Pary, M., J. Raynauld, and J.G. Simonao (989) Technical Change as an Unobservable Sochasic Variable: an Applicaion o he U.S. Primary-Meals Indusry, Working paper, École des HEC. Ruud, P.A. (99) Exensions of esimaion mehods using he EM algorihm, Journal of Economerics, 49, Slade, M. (989) Modeling sochasic and cyclical componens of echnical change: an applicaion of he kalman filer, Journal of Economerics, 4, Shumway, R. H. (988) Applied Saisical Time Series Analysis, Prenice Hall, Englewood Cliffs. Shumway, R.H. and D.S. Soffer (982) An Approach o Time Series Smoohing and Forecasing using he EM Algorihm, Journal of Time Series Analysis, 3, Sock, J.H. and M.W. Wason (99) A probabiliy model of he coinciden economic indicaors, in Lahiri e G.H. Moore, ed., Leading Economic Indicaors, Cambridge Universiy Press, Cambridge, Wason, M. and R. Engle (983), Alernaive Algorihms for he Esimaion of Dynamic Facor, Mimic and Varying Coefficien Regression Models, Journal of Economerics,

9 APPENDIX : DEMO.GAU NEW;.7 Sae space marices' dimensions N=3; M=4; NM=; G=4; K=; K2=; NPAR=; NOBS=79; Include file wih he CONSTRUC procedure #INCLUDE Daa reading and ransformaion LOAD Y[79,]=EMP.prn; LOAD Y2[79,]=SHIP.prn; LOAD Y3[79,]=RTS.prn; Y=Y~Y2~Y3; X = zeros(nobs,) ; X2 = zeros(nobs,) ; z = ime-varying marix Z (filled wih zeros if Names of he oupu files FOR THE EM FOR THE Iniial values DELTA={

10 @ Algorihm choice variables N, M, NM, G, K and K2 correspond o he sae-space marices dimensions defined earlier.!npar is he number of "free" parameers. (number of parameers o esimae) CHOICE = = o use he EM algorihm is he number of observaions available for he observed variables. (Ys and =2 o use he SCORING =3 o use boh EM and SCORING observed variable is loaded and used o form he (NOBS by N) marix Y.!Noice ha we filled X and X2 wih zeros because here is no exogenous variables in our EM algorihm opions OPTION=ZEROS(5,);!See Appendix 4 for furher deails on seing up a ime-varying Z marix.!all resuls afer he final ieraion are sored in separae files.!i's a good idea o change hese names for each new model because he program will overwrie hese files.!delta is he (NPAR by ) vecor of parameers o esimae.!i will be used by he <name>.prc file which build he model's srucure.!we mus give iniial values for his vecor.!some models can be very sensiive o iniial values. Choosing an "adequae" se can help convergence. Convergence Maximum number of = o skip he convergence = To prin he unobservable variables a he = To prin A and P a each = To prin sae space model a each = To prin he difference in he vecor of = To prin he variances of he unobservable = To prin he smoohed = number of residuals' auocorrelaion o compue = = o use he smoohed esimaes of a() as saring value for all ieraions bu he firs = o use as a saring value for = o use he smoohed esimaes of P() as saring value for all he ieraions bu he firs = o use he auomaic calculaion of he marix P based upon he model's = n o compue he sandard errors afer he n h = m o compue he coefficien wih a "hill climbing" mehod afer he m SCORING algorihm opions ells he program he name of he.prc file.!each new model need a new.prc file. OPT=ZEROS(,); convergence maximum number of = if you wan o skip he ieraions' = o have he smoohed esimaes a he = o have residuals and = if he unobservable variable is saionnary = if

11 = o use he numerical derivaives = o use he analyical = if you wan he iniial covariance marix P o be compued by he program using he model's = if Z is ime-varying. The ime-varying marix has o be provided by he marix Convergence crierion on gradien vecor (greaes gradien CHOICE=3, he program will firs sar wih he EM algorihm unil he maximum number of ieraions is reached or unil convergence occured. Then, he esimaion will proceed wih he scoring algorihm.!noe ha if a VCV marix become non-posiive definie while using he scoring algorihm, he program will go back o he EM algorihm and sar he process all over.!opions for he EM and he scoring algorihm are chosen separaely.!in his case, here will be 5 ieraions wih he EM algorihm before hose wih he scoring algorihm.!opion 4 and 5 are no currenly acive.

12 !This opion is srongly recommended. For more informaion, see he.prc file. APPENDIX 2: DEMO.PRC PROC ()=CONSTRUC(DELTA); LOCAL Iniializing marices LOAD SZ = SIZE.FMT; N=SZ[,]; M=SZ[2,]; K=SZ[3,]; K2=SZ[4,]; NM=SZ[5,]; G=SZ[6,]; A=ZEROS(M,); P=ZEROS(M,M); Z=ZEROS(N,M); BETA=ZEROS(N,K); S=ZEROS(N,NM); H=ZEROS(NM,NM); T=ZEROS(M,M); GAM =ZEROS(M,K2); R=ZEROS(M,G) ; Building sae-space marices wih he vecor DELTA Z[,]=DELTA[,]; Z[2,]=DELTA[2,]; Z[3,]=DELTA[3,]; Z[,2]=.; Z[2,3]=.; Z[3,4]=.; T[,]=DELTA[4,]; T[2,2]=DELTA[5,]; T[3,3]=DELTA[6,]; T[4,4]=DELTA[7,]; R=EYE(4); Q[,]=.; Q[2,2]=DELTA[8,]; Q[3,3]=DELTA[9,]; Compuing P marix according o he model's srucure (see opions) IF SZ[9,] == ; RQR=R*Q*R'; VP=INV(EYE(M*M)-T.*.T)*VEC(RQR); I=; P=ZEROS(M,M); DO WHILE I <= M ;

13 P[.,I] = VP[(M*(I-)+):(M*I),]; I=I+; ENDO ; ELSE; IF ENDIF; RETP(A,P,Z,BETA,S,H,T,GAM,R,Q) ; ENDP;!This is he only par of his file ha need o be changed for each new model.!i gives he conen of each marix in he model.!each marix is iniialized o have he correc size and is filled wih zeros. One only need o assign o each para-meer o esimae, he corresponding elemen of he vecor DELTA and fix he value of any consrained parameer.!as we can see, he firs hree elemens of vecor DELTA he parameers in marix Z which links observed variables o he common facor EA.!The four nex elemens correspond o he auoregressive coefficiens of he sae variables. (marix T)!The 3 las elemens of DELTA are he variances associaed wih each idiosyncraic componen's random disurbance.!the variance of he common facor is se o and so are he parameers linking idiosyncraic componens and observed variables in marix Z.!Leing he program o compue he P marix is he bes way o impose only a diffuse prior. The P marix will hen be consisen wih he model's srucure and he given iniial values. This is conroled by selecing he corresponding opion in he <name>.gau file.!if in any case one wan o provide his "own" P marix, his should be done on his line.

14 APPENDIX 3: NON-STATIONNARY EXAMPLES We simulaed he following models: Firs model. [ ] U U V = Y - 8 u v U U V = U U V - 9 wih.4.25 = Q. Second model. u 2 u + C C.7. - = 2 Y Y [ ] e + C C = C C - 2wih [ ].9.63 H =. = Q 3 Esimaed parameers are in bold ypeface. The firs example will be found in he file WAT.GAU and he second in he file NONSTAT.GAU. The resuls obained can respecively be found in he files WAT_SC.RES and NSTAT_SC.RES. For each model, he resuls can be summed up by Firs model.

15 ITERATION # COEFFICIENTS VECTOR ITERATION GRADIENT T-TEST PARAM. K K- K- COORD T(2,2) T(2,3) GAM(,) Q(,) LIKELIHOOD ITERATION ITERATION DIFFERENCE K K Second model. ITERATION # COEFFICIENTS VECTOR ITERATION GRADIENT T-TEST PARAM. K K- K- COORD Z(,) Z(2,) H(,) H(2,2) T(,) T(,2) LIKELIHOOD ITERATION ITERATION DIFFERENCE K K

16 Some observaions need o be made abou he resuls. Firs, noice ha only four parameers are esimaed in he firs model. For idenificaion concerns, we need o fix he variance Q(2,2) o.4 in he equaion for U even if Z(,2) is se o.. This is due o he special naure of he model. There is only one observable variable (Y) o idenify wo disinc sources of random disurbances, namely u and v. If Q(2,2) is no fixed, he algorihm will have a hard ime figuring ou wheher he randomness of Y comes from u or v. Acually, i will push eiher he value of Q(,) or he value of Q(2,2) owards zero so as o "eliminae" one of he wo random disurbances. In he process, one variance may even become negaive! This is a common resul when an idenificaion problem occurs. Second, noice ha he esimaed values for Z(,) and Z(2,) in he second model don' quie mach he simulaed values. This is because we need o consider hese values on a relaive basis, ha is Z(2,)/Z(,). If we compue his raio we obain.768, a value much closer o he "simulaed raio" of.7. I also worhs noing ha he sum of he wo auoregressive coefficiens in he equaion for C is more significan han he absolue values of hese coefficiens. In his precise case, he "esimaed sum" is abou.98 and he "simulaed sum" is.96. For he firs example, he simulaed sum of coefficien was.96 and he esimaion gave us a sum of.932.

17 APPENDIX 4: TIME VARYING Z MATRIX The following model assumes a ime varying Z marix: Y = Z, α + Β X + e α = α - + n. The SCOREM 2. imevz.gau file Sae space marices' dimensions N=; M=; NM=; G=; K=3; K2=; NPAR=4; Daa reading and ransformaion LOAD Y[26,]=VOBS.PRN; LOAD X[26,3]=EXO.PRN; x2 = zeros(nobs,) ; load Names of he oupu files @ OUTPUT FILE FOR THE EM OUTPUT FILE FOR THE Saring values (opionnal if scoring is no used alone) IF YOU DON'T WANT TO GIVE STARTING LET DELTA[4,]=.7.3

18 Include file wih he CONSTRUC procedure #INCLUDE THE NAME OF THIS FILE HAS TO BE CHANGED Algorihm choice CHOICE = = o use he EM =2 o use he SCORING =3 o use he inegraed SCORING algorihm opions OPT=ZEROS(,); convergence op[2,]= ieraions op[3,]= = if you wan o skip he ieraions' op[5,]= = o have he smoohed esimaes a he op[6,]= = o have residuals and op[4,]= = if he unobservable variable is saionnary = if op[7,]= = o use he numerical derivaives = o use he analyical op[8,]= = if you wan he iniial covariance marix P o be calculaed by he IF OPT[4,]==; he use of analyical derivaives is proscribed if he unobservable variable is no ENDIF; = if Z is ime-varying. The ime-varying marix has o be provided by he marix Convergence crierion on gradien vecor (greaes gradien The SCOREM 2. imevz.prc file is: PROC ()=CONSTRUC(DELTA) Variables locales

19 LOCAL Former les marice du modele sae-space LOAD SZ = SIZE.FMT; N=SZ[,]; M=SZ[2,]; K=SZ[3,]; K2=SZ[4,]; NM=SZ[5,]; G=SZ[6,]; A=ZEROS(M,); P=ZEROS(M,M); a=-2; Z=ZEROS(N,M); BETA=ZEROS(N,K); S=ZEROS(N,NM); H=ZEROS(NM,NM); T=ZEROS(M,M); GAM =ZEROS(M,K2); R=ZEROS(M,G) ; Q=ZEROS(G,G); BETA[,]=DELTA[,]; BETA[,2]=DELTA[2,]; BETA[,3]=dela[3,]; H[,]=.; T[,]=.; Q[,]=dela[4,]; S[,]=.; COMPUTES P BASED UPON THE MODEL'S IF SZ[9,] == ; RQR=R*Q*R'; VP=INV(EYE(M*M)-T.*.T)*VEC(RQR); I=; P=ZEROS(M,M); DO WHILE I <= M ; P[.,I] = VP[(M*(I-)+):(M*I),]; I=I+; ENDO ; ELSE; IF ENDIF; RETP(A,P,Z,BETA,S,H,T,GAM,R,Q) ; ENDP; Noe: For idenificaion purposes, he variance of he measuremen equaion has been fixed o.. This value was esimaed afer fiing an AR() model on he Y variable VOBS.