Exercses (Part 4) Introducton to R UCLA/CCPR John Fox, February 2005 1. A challengng problem: Iterated weghted least squares (IWLS) s a standard method of fttng generalzed lnear models to data. As descrbed n Secton 5.5 of Fox, An R and S- PLUS Companon to Appled Regresson, the IWLS algorthm appled to bnomal logstc regresson proceeds as follows: (a) Set the regresson coeffcents to ntal values, such as β (0) = 0 (where the superscrpt 0 ndcates start values). (b) At each teraton t calculate the current ftted probabltes µ, varance-functon values ν, workng-response values z, andweghtsw: µ (t) = [1+exp( η (t) )] 1 v (t) = µ (t) (1 µ (t) ) z (t) = η (t) +(y µ (t) )/v (t) w (t) = n v Here, n represents the bnomal denomnator for the th observaton; for bnary data, allofthen are 1. (c) Regress the workng response on the predctors by weghted least squares, mnmzng the weghted resdual sum of squares nx =1 w (t) (z (t) x 0 β) 2 where x 0 s the th row of the model matrx. (d) Repeat steps 2 and 3 untl the regresson coeffcents stablze at the maxmum-lkelhood estmator β. b (e) Calculate the estmated asymptotc covarance matrx of the coeffcents as bv( b β)=(x 0 WX) 1 where W = dag{w s the dagonal matrx of weghts from the last teraton and X s the model matrx. Problem: Program ths method n R. The functon that you defne should take (at least) three arguments: The model matrx X; the response vector of observed proportons y; and 1
the vector of bnomal denomnators n. Isuggestthatyouletn default to a vector of 1s (.e., for bnary data, where y conssts of 0s and1s), and that you attach a column of 1s tothe model matrx for the regresson constant so that the user does not have to do ths when the functon s called. Programmng hnts: Adapt the structure of the example developed on pages 273 274 of Fox (but note that the example s for bnary logstc regresson, whle the current exercse s to program the more general bnomal logt model). Use the lsft functon to get the weghted-least-squares ft, callng the functon as lsft(x, z, w, ntercept=false), wherex s the model matrx; z s the current workng response; and w s the current weght vector. The argument ntercept=false s needed because the model matrx already has a column of 1s. The functon lsft returns a lst, wth element $coef contanng the regresson coeffcents. See help(lsft) for detals. One trcky pont s that lsft requres that the weghts (w) beavector, whle your calculaton wll probably produce a one-column matrx of weghts. You can coerce the weghts to a vector usng the functon as.vector. Return a lst wth the maxmum-lkelhood estmates of the coeffcents, the covarance matrx of the coeffcents, and the number of teratons requred. You can test your functon on the Mroz data n car, beng careful to make all of the varables numerc (as on page 275). You mght also try fttng a bnomal (as opposed to bnary) logt model. 2. Another challengng problem (though perhaps somewhat less so): Amatrxssadtoben (reduced ) row-echelon form when t satsfes the followng crtera: (a) All of ts nonzero rows (f any) precede all of ts zero rows (f any). (b) The frst entry (from left to rght) called the leadng entry n each nonzero row s 1. (c) The leadng entry n each nonzero row after the frststotherghtoftheleadngentry n the prevous row. (d) All other entres are 0 n a column contanng a leadng entry. A matrx can be put nto row echelon form by a sequence of elementary row operatons, whch are of three types: (a) Multply each entry n a row by a nonzero constant. (b) Add a multple of one row to another, replacng the other row. (c) Exchange two rows. Gaussan elmnaton s a method for reducng a matrx to row-echelon form by elementary row operatons. Startng at the frst row and frst column of the matrx, and proceedng down and to the rght: 2
(a) If there s a 0 n the current row and column (called the pvot), f possble exchange for a lower row to brng a nonzero element nto the pvot poston; f there s no nonzero pvot avalable, move to the rght and repeat ths step. If there are no nonzero elements anywhere to the rght (and below), then stop. (b) Dvde the current row by the pvot, puttng a 1 n the pvot poston. (c) Proceedng through the other rows of the matrx, multply the pvot row by the element n the pvot column n another row, subtractng the result from the other row; ths zeroes out the pvot column. Consder the followng example: 2 0 1 2 4 0 1 0 Dvde row 1 by -2: Subtract 4 row 1 from row 2: Subtract 6 row 1 from row 3: Multply row 2 by -1: Subtract 0.5 row 2 from row 1: 4 0 1 0 0 0 1 4 0 0 1 4 0 0 2 8 0 0 2 8 0 0 2 8 Add 2 row 2 to row 3: 0 0 0 0 The matrx s now n row-echelon form. The rank of a matrx s the number of nonzero rows n ts row-echelon form, and so the matrx n ths example s of rank 2. Problem: Wrte an R functon to calculate the row-echelon form of a matrx by elmnaton. Programmng hnts: 3
When you do floatng-pont arthmetc on a computer, there are almost always roundng errors. One consequence s that you cannot rely on a number beng exactly equal to a value such as 0. When you test that an element, say x, s0,therefore,youshoulddo so wthn a tolerance e.g., x < 1 10 6. The computatons tend to be more accurate f the absolute values of the pvots are as large as possble. Consequently, you can exchange a row for a lower one to get a larger pvot even f the element n the pvot poston s nonzero. 3. A less dffcult problem: Wrte a functon to compute runnng medans. Runnng medans are a smple smoothng method usually appled to tme-seres. For example, for the numbers 7, 5, 2, 8, 5, 5, 9, 4, 7, 8, the runnng medans of length 3 are 5, 5, 5, 5, 5, 5, 7, 7. The frst runnng medan s the medan of the three numbers 7, 5, and 2; the second runnng medan s the medan of 5, 2, and 8; and so on. Your functon should take two arguments: the data (say, x), and the number of observatons for each medan (say, length). Notce that there are fewer runnng medans than observatons. How many fewer? 4. Debuggng Functons: Here are solutons to programmng problems 1 through 3, but each functon has a bug (or two) that ether causes t to fal or, possbly only n certan crcumstances, to gve the wrong answer. In each case, fnd the bug(s) and fx the functon. A fle wth the bugged functons s avalable for download from the course web ste. a. A functon to calculate logstc-regresson estmates by teratvely reweghted least-squares: lregiwls <- functon(x, y, n=rep(1,length(y)), maxiter=10, tol=1e-6){ # bugged! # X s the model matrx # y s the response vector of observed proporton # n s the vector of bnomal counts # maxiter s the maxmum number of teratons # tol s a convergence crteron X <- cbnd(1, X) # add constant b <- blast <- rep(0, ncol(x)) # ntalze t <- 1 # teraton ndex whle (t <= maxiter){ f (max(abs(b - blast)/(abs(blast) + 0.01*tol)) < tol) break eta <- X %*% b mu <- 1/(1 + exp(-eta)) nu <- as.vector(mu*(1 - mu)) w <- n*nu z <- eta + (y - mu)/nu b <- lsft(x, z, w, ntercept=false)$coef blast <- b t <- t + 1 # ncrement ndex f (t > maxiter) warnng( maxmum teratons exceeded ) Vb <- solve(t(x) %*% dag(w) %*% X) lst(coeffcents=b, var=vb, teratons=t) 4
b. A functon to compute the row-echelon form of a matrx by Gaussan elmnaton: rowechelonform <- functon(a){ # bugged! n <- nrow(a) m <- ncol(a) for ( n 1:mn(c(m, n))){ currentcolumn <- A[,] currentcolumn[1:n < ] <- 0 whch <- whch.max(abs(currentcolumn)) # fnd maxmum pvot n current # column at or below current row pvot <- A[whch, ] f (abs(pvot) == 0) next # check for 0 pvot f (whch > ) A[c(, whch),] <- A[c(whch, ),] # exchange rows A[,] <- A[,]/pvot # pvot row <- A[,] A <- A - outer(a[,], row) # sweep A[,] <- row # restore current row for ( n 1:n) f (max(abs(a[,1:m])) == 0) A[c(,n),] <- A[c(n,),] # 0 rows to bottom A Note that ths functon returns the rght answer for the matrx used as an example n Problem 2, 2 0 1 2 4 0 1 0 whose row-echelon form s but gves the wrong answer for the matrx 0 0 0 0 16 2 3 13 5 11 10 8 9 7 6 12 4 14 15 1 whose correct row-echelon form s 0 1 0 3 0 0 1 3 0 0 0 0 5
c. A functon to calculate runnng medans: runnngmedan <- functon(x, length=3){ # bugged! # x: a numerc vector # length: the number of values for each runnng medan, defaults to 3 n <- length(x) X <- matrx(x, n, length) for ( n 1:length) X[1:(n - + 1), ] <- x[-(1:( - 1))] apply(x, 1, medan)[1:(n - length + 1)] 2. Object-Orented Programmng: Modfy your logstc-regresson functon from Problem 1 so that t returns an object of class lreg, whch ncludes components for the coeffcents, ther covarance matrx, and the resdual devance, along wth the number of observatons and the number of teratons requred to maxmze the lkelhood. a. Wrte methods for the generc functons summary, prnt, coef, vcov, anddevance, to prnt model a model summary, to prnt a bref report, and to return the logstcregresson coeffcents, ther covarance matrx, and the resdual devance for objects of ths class. b. More ambtously, wrte a method for the generc functon anova to compare two objects of class lreg by a lkelhood-rato test, supposng that one object represents a model that s properly nested wthn the other. Allow the larger model to be gven ether frst or second, and try to check that the models are n fact nested. 6