Number of Parameters Counting in a Hierarchically Multiple Regression Model

Size: px

Start display at page:

Download "Number of Parameters Counting in a Hierarchically Multiple Regression Model"

Frederica Neal
5 years ago
Views:

1 DOI: 0.7/scntl Number of Parameters Countng n a Herarchcally Multple Regresson Model H.J. Zanodn, Noran Abdullah and S.J. Yap School of Scence and Technology, Unverst Malaysa Sabah, Kota Knabalu, Sabah, 88400, Malaysa ABSTRACT Background: Manually, when a dependent varable s affected by a large number of ndependent varables, the number of parameters helps researchers determne the number of ndependent varables to be consdered n an analyss. However, when there are many parameters to be estmated n a model, the manual countng s tedous and tme consumng. Thus, ths study derves a method to determne the number of parameters systematcally n a model. Methods: The model buldng procedure n ths study nvolves removng varables due to multcollnearty and nsgnfcant varables. Eventually, a selected model s obtaned wth sgnfcant varables. Results: The fndngs of ths study would enable researchers to count the number of parameters n a resultng model (selected model) wth ease and speed. On top of that, models whch fulfll the assumpton are consdered n the statstcal analyss. In addton, human errors caused by manual countng can also be mnmsed and avoded by mplementng the proposed procedure. Concluson: These fndngs wll also undoubtedly help many researchers save tme when ther analyses nvolve complex teratons. Key words: Number of parameters, sngle quanttatve ndependent varable, dummy varable, nteracton varable, herarchcally multple regresson model Scence Internatonal (): 7-4, 04 INTRODUCTION Accordng to Ramanathan n a lnear regresson model, regresson coeffcents are the unknown parameters to be estmated. In a smple lnear regresson, only two unknown parameters have to be estmated. However, problems arse n a multple lnear regresson, when the numbers of parameters n the model are large and more complex, where three or more unknown parameters are to be estmated. Challenges arose when computer programme has to be wrtten and complex teratons are requred to perform wth certan crteron. Thus, the exact number of parameters nvolved should be known n order to prepare the amount of data to sut such large and complex model. It s also mportant to note that n order to have a unque soluton n fndng the estmated parameters, accordng to the assumptons of multple regresson model stated by Gujarat and Porter, the number of estmated parameters must be less than the total number of observatons. 4 Accordng to Zanodn et al., Yahaya et al. n a multple lnear regresson analyss, there are four phases n gettng the best model, namely: Lstng out all possble models, gettng selected models, gettng best model and conductng the valdty of goodness-of-ft. In phase, the number of parameters for a possble model s denoted by Correspondng Author: H.J. Zanodn, School of Scence and Technology, Unverst Malaysa Sabah, Kota Knabalu, Sabah, 88400, Malaysa NP. To get the selected models, after lstng out all of the possble models n phase, multcollnearty test and coeffcent test are conducted on the possble models n phase. Before contnung to phase, number of parameters n each model must be less than the sample sze, n. Dscard the model whch faled the ntal crtera. In the multcollnearty test, multcollnearty source varables are removed from each of the possble models. Then, coeffcent test s conducted on the possble models that are free from multcollnearty problem. Detaled procedure of ths phase s explaned n Zanodn et al.. Ths s to elmnate nsgnfcant varables from each of the possble models. For a general model Ma.b.c wth parent model number a, the number of varables removed due to multcollnearty problem s denoted by b, the number of varables elmnated due to nsgnfcance s denoted by (k+) and the resultng number of parameters for a selected model s represented by (k+). In most of the cases, f the number of the unknown parameters to be estmated for a possble model s large, then the number of parameters for a selected model wll most probably be 5,6 large too. In these cases, the manual countng on the large number of parameters s found to be tme consumng. Furthermore, some of the parameters mght be mssed out due to human error n manual countng. Thus, the objectve of ths study s to propose a method 04 Scence Internatonal 7

2 to count the number of parameters for a selected model, parameters to be estmated, and are sngle (k+). The nformaton of the NP, b and c s useful n quanttatve ndependent varables, s frst-order gettng the number of parameters for a selected model, nteracton varable, D s sngle ndependent dummy (k+). varable, D s frst-order nteracton varable of and Accordng to Gujarat and Porter, a smple lnear regresson model wthout any nteracton varable can be wrtten as follows: Y = $ +$ +u () 0 where, Y s dependent varable, $ 0 and $ are regresson coeffcents and they are the unknown parameters to be estmated, s sngle quanttatve ndependent varable and u s error term. So, t can be observed that there are two unknown parameters to be estmated n Eq.. Ths equaton can also be wrtten n the form as follows: Y = f( ) Next, a herarchcally multple lnear regresson 7 models wth nteracton varable can be wrtten as follows: Y = $ +$ +$ +$ +u () 0 where, Y s dependent varable, $ 0, $, $ and $ are regresson coeffcents and they are the unknown parameters to be estmated, and are sngle quanttatve ndependent varables, s frst-order nteracton varable and u s error term. Thus, t can be seen that there are four unknown parameters to be estmated n Eq.. Ths equaton can also be wrtten n the followng form: Y = f(,, ) Next, an example for a lnear regresson model wth nteracton varables and dummy varables s shown as follows: Y = $ +$ +$ +$ +$ D+$ D+$ D+u 0 D D D () where, Y s dependent varable, $, $, $ $, $, $ and 0 D D $ are regresson coeffcents and they are the unknown D D, D s frst-order nteracton varable of and D and u s error term. Therefore, t can be observed that there are seven unknown parameters to be estmated n Eq.. Ths equaton can also be wrtten as follows: Y = f(,,, D, D, D) Equaton - can be wrtten n general model n the form: Y = S +S W +S W S W +u (4) 0 k k where, S 0 s the ntercept and S j s the jth partal regresson coeffcent of the correspondng ndependent varable W j for j =,,..., k. Accordng to Zanodn et al., ndependent varable W j ncluded the sngle ndependent varables, nteracton varables, generated varables, dummy varables and transformed varables. In ths study, (k+) denotes number of parameters for a selected model. The correspondng labels of the general model n Eq. 4- are shown n Table. From Table, t s known that $ represents the S 0 0 n the general model, $ represents the S and the same goes to other estmated parameters n Table. Varable n Eq. represents W n the general model and the same goes to other varables n Table. Instead of countng the number of parameters one by one as above, a equaton to count the number of parameters n model wthout nteracton varable and n model wth nteracton varable s proposed n ths study. The Eq. 5 s presented as follows: g+ h +,v = 0 NP = v+ = C + (g+ )h+,v =,,... (5) where, NP s number of parameters for a possble model, g s number of sngle ndependent quanttatve varables, h s number of sngle ndependent dummy varables and v s hghest order of nteracton (between sngle Table : Correspondng labels of general model and Eq. Coeffcents Independent varables No. of ndependent varables (k) General model Eq. General model Eq. 0 S 0 $ S $ W S $ W S $ W 4 S $ W D 4 D 4 5 S $ W D 5 D 5 6 S $ W D 6 D 6 04 Scence Internatonal 8

3 ndependent quanttatve varable) n the model. Here, NP = g+h+ = 7 v = 0 denotes model wthout nteracton varable (or model wth zero-order nteracton varable) and For smplcty, consder another example for v =,, denotes model wth frst or hgher order calculatng the number of parameters n a model wthout nteracton varable(s). Hence, Eq. 5 can now be tested n an nteracton varable. Consderng Eq. 7 whch s a the followng nstances to prove ts valdty n countng model wth zero-order nteracton varable (or v equals the number of parameters n a herarchcally multple to 0), number of sngle quanttatve ndependent varable, regresson model. The am of ths study s to propose a g equals to 8 and number of sngle dummy varable, h method to count the number of parameters for a selected equals to 0, as follows: model, (k+). The nformaton of the NP, b and c s useful n gettng the number of parameters for a selected Y = f(,,, 4, 5, 6, 7, 8, B, C, L, E, model, (k+). W, K, A, G, H, S) (7) MATERIALS AND METHODS where,,,, 4, 5, 6, 7 and 8 are sngle Ths wll help to better understand the applcaton quanttatve ndependent varables and B, C, L, E, W, K, of the equaton as mentoned n prevous secton. In A, G, H and S are 0 sngle dummy varables. Then, the order to acheve a model free from multcollnearty followng s obtaned: effects and nsgnfcant effects the followng 4 phase model buldng procedure s mplemented (detals can NP = g+h+ = 9, 4 be found n ). Models wth nteracton varable: Now, consder C Phase : All possble models models wth nteracton varable (.e., v =,, ) n ths C Phase : Selected model subsecton. Accordng to the equaton mentoned n Multcollnearty test and coeffcent test (Include Eq. 5, t s known that the number of parameters n a NPM s Near Perfect Multcollnearty test and model wth nteracton varable s calculated n a NPC s Near Perfect Collnearty test) dfferent way from a model wthout nteracton varable. C Phase : Best model A few examples are presented to provde better C Phase 4: Goodness-of-ft understandng of ths equaton. For nstance, model wth nteracton varable up to frst-order (.e., v equals to ), Randomness test and normalty test: Ths study also number of sngle quanttatve ndependent varable, g revealed the parameters of ndependent varables measure equals to and number of sngle dummy varable, h multcollnearty effect between ndependent varables equals to 5 s presented n Eq. 8: parameters and structural parameters. As dscussed n earler secton, the number of parameters s very mportant before arrvng at a selected and best model. Y = f(, 4, 4, D, B, R, A, G, D, B, R, A, Thus, some llustratons follow: G, D, B, R, A, G) (8) Models wthout nteracton varable: Here, the followng models are consdered wthout nteracton varable or wth zero-order nteracton varable. As ponted out earler n Eq. 5, the number of parameters n the case of model wthout nteracton varable (or v = 0) can be computed usng g+h+. For nstance, consder Eq. 6, a model wth zero-order nteracton varable (or v equals to 0), number of sngle quanttatve ndependent varable, g equals to and number of sngle dummy varable, h equals to 5, as follows: Y = f(, D, B, R, A, G) (6) where, Y s a dependent varable, s a sngle quanttatve ndependent varable (g =) and D, B, R, A and G are 5 sngle dummy varables (h =5). Then, total number of parameters nvolved s: where, and are sngle quanttatve ndependent 4 varables, D, B, R, A and G are sngle dummy varables and, D, B, R, A, G, D, B, R, A and G are frst-order nteracton varables. Then, ths led 4 to: v+ g NP = C + [(g + )h+ ] = ( ) = ( ) + = = C + [(+ )5+ ] = C + 6 = C + C + 6 = 9 (Total No.of parameters as n Eq.8) Next, consder a larger model wth hgher order of nteracton varable, for nstance, a model wth ffth-order nteracton (.e., v equals to 5), number of 04 Scence Internatonal 9

4 sngle quanttatve ndependent varable, g equals to 6 and Y = f(,,, 4, 5, 6, 7,,, 4, 5, 6, number of sngle dummy varable, h equals to 5 as 7,, 4, 5, 6, 7, 4, 5, 6, 7, 45, 46, 47, presented n Eq. 9: 56, 57, 67,, 4, 5, 6, 7, 4, 5, 6, 7, 45, 46, 47, 56, 57, 67, 4, 5, 6, 7, Y = f(,,, 4, 5, 6,,, 4, 5, 6,, 4, 45, 46, 47, 56, 57, 67, 45, 46, 47, 56, 5, 6, 4, 5, 6, 45, 46, 56,, 4, 5, 57, 67, 456, 457, 467, 567, 4, 5, 6, 7, 45, 6, 4, 5, 6, 45, 46, 56, 4, 5, 6, 46, 47, 56, 57, 67, 45, 46, 47, 56, 46, 56, 45, 46, 56, 456, 4, 5, 6, 45, 57, 67, 456, 457, 467, 567, 45, 46, 47, 56, 46, 56, 45, 46, 56, 456, 45, 46, 56, 456, 57, 67, 456, 457, 467, 567, 456, 457, 467, 456, 45, 46, 56, 456, 456, 456, 456, D, B, 567, 4567, 45, 46, 47, 56, 57, 67, 456, R, A, G, D, B, R, A, G, D, B, R, 457, 467, 567, 456, 457, 467, 567, 4567, 456, A, G, D, B, R, A, G, 4 D, 4B, 457, 467, 567, 4567, 4567,, 457, 4 R, 4 A, 4 G, 5 D, 5 B, 5 R, 5 A, 5 G, 6D, 467, 567, 4567, 4567, 4567, 4567, B, C, E, W, K, 6 B, 6 R, 6 A, 6G) (9) A, G, H, S, B, C, E, W, K, A, G, H, S, B, C, E, W, K, A, G, H, S, In Eq. 9,,,, 4, 5 and 6 are sngle B, C, E, W, K, A, G, H, S, 4B, quanttatve ndependent varables,,, 4, 5, 6, 4 C, 4 E, 4 W, 4 K, 4 A, 4 G, 4 H, 4 S, 5 B, 5C,, 4, 5, 6, 4, 5, 6, 45, 46 56, D, B, R, 5 E, 5 W, 5 K, 5 A, 5 G, 5 H, 5 S, 6 B, 6 C, 6E, A, G, D, B, R, A, G, D, B, R, A, 6 W, 6 K, 6 A, 6 G, 6 H, 6 S, 7 B, 7 C, 7 E, 7W, G, 4 D, 4 B, 4 R, 4 A, 4 G, 5 D, 5 B, 5 R, 5 A, 5G, 7 K, 7 A, 7 G, 7 H, 7S) (0) 6 D, 6 B, 6 R, 6 A and 6G are frst-order nteracton varables,, 4, 5, 6, 4, 5, 6, 45, 46, 56, Here, 456, 457, 467, 567, 4567, 4567 and , 5, 6, 45, 46, 56, 45, 46, 56 and 456 are are 5th order nteracton varables and 4567 s a 6th order second-order nteracton varables, 4, 5, 6, 45, nteracton varable. Then, total number of parameters:,,,,,,,,, and are thrd-order nteracton varables,,, ,, and are fourth-order nteracton varables and s ffth-order nteracton varable. 456 Then, total number of parameters: v+ g = = NP = C + [(g + )h + ] = C + [(6+ )5+ ] 6 6 ( ) = = C + 6 = 99 Lastly, consder another example of a larger model whch has nteracton varable up to 6th order and nne sngle dummy varables. Consder Eq., a model wth hghest order of nteracton, v equals to 6, number of sngle quanttatve ndependent varable, g equals to 7 and number of sngle dummy varable, h equals to 9. v+ g = NP = C + [(g + )h+ ] 7 ( ) 6 + = = C ( ) = C + C + C + C + C + C + C + 7 = 00 () As can be seen from the llustratons, number of parameters calculated usng the derved equaton tally wth that n manual countng. RESULTS AND DISCUSSION The proposed equaton defned n Eq. 5 s especally useful n counter checkng the varables when lstng all of the possble models n an analyss. Ths s because some of the varables mght be mssed out when there are a large number of parameters nvolved n a possble model. Table shows all the possble models for an Table : No. of parameters n each of all possble models All possble models NP g h v M: Y = f(, D) 0 M: Y = f(, D) 0 M: Y = f(, D) 0 M4: Y = f(,, D) 4 0 M5: Y = f(,, D) 4 0 M6: Y = f(,, D) 4 0 M7: Y = f(,,, D) 5 0 M8: Y = f(,,, D, D, D) 7 M9: Y = f(,,, D, D, D) 7 M0: Y = f(,,, D, D, D) 7 M: Y = f(,,,,,, D, D, D, D) M: Y = f(,,,,,,, D, D, D, D) 04 Scence Internatonal 40

5 analyss that has two sngle quanttatve ndependent Table 6 shows that model M. s free from varables ( and ) and one sngle dummy varable (D). multcollnearty source varables because all the absolute In Table, wth the nformaton on g, h and v, the correlaton coeffcent values between all the NP for possble models M-M can be computed by ndependent varables are less than (except the usng Eq. 5. Then, the number of parameters for each of dagonal values). the possble models can be counterchecked by usng the By observng the model M., t s found that the computed NP values. For smplcty, each of the number of varables removed due to multcollnearty models can be wrtten n general form as n Eq. 4. problem s. Therefore, b for model M. s. More After ntroducng the equaton n countng the detals on the defnton of model name can be found n number of parameters for a possble model, the way of 4,6,0 and. Thus the resultng model, M. s free from gettng the number of parameters for a selected model, multcollnearty effects and the coeffcent test s 8,9 (k+) s presented. A model M, s used as an conducted on the model. The task then s to elmnate llustraton and t had also been mentoned earler n nsgnfcant varables from ths model, M.. Eq.. So, multcollnearty test s conducted on the In Table 7, t s found that varable has the hghest possble model, model M. The removal of p-value among other ndependent varables and s greater multcollnearty source varables from ths model are than 0.05 (snce the number of sngle quanttatve shown n Table -4. Ths study uses the modfed ndependent varables s greater than 5 and the coeffcent method n removng multcollnearty source varables test s a two-tal, the level of sgnfcance s set at 0%. (excel command: COUNTIF()). Ths s based on recommendatons). Thus, varable In Table, all of the multcollnearty source s elmnated from model M. and ths reduced model varables (varables wth absolute correlaton coeffcent s called model M.. as varable s elmnated due to values greater than or equal to ) are crcled. Then, nsgnfcance. Detals on the coeffcent test can be 8,, t s found that varables, D, D and D have found n. frequences. So, accordng to, model M belongs Table 8 shows that model M.. s free from to case B. To avod confuson between dummy nsgnfcant varable because both the p-values of varable B and case B, case B s represented by case n varable and D are less than From Table 8, t s ths study. found that parameters (constant of the model, Smlarly, case C s represented by case n ths coeffcent of varables and D) are left n model study. Based on the removal steps for case, varable M.., or n other words, (k+) equals to. From the whch has the weakest absolute correlaton coeffcent model name, model M..; t s notced that varable wth dependent varable Y, f compared to varables D, s elmnated n coeffcent test and c equals to. As D and D, s removed from model M. The same mentoned earler n Eq., there are seven unknown removal steps are carred out on the reduced model; parameters to be estmated for model M, so NP equals model M., as presented n Table 4. to 7. Thus, by knowng the NP, b and c for model After removng varable D from model M. n M.. (.e., Ma.b.c), the number of parameters (k+) Table 4, detals correlaton coeffcents of the reduced for model M.. can be counterchecked usng the model M. are shown n Table 5. It s observed that proposed equatonn ths study as: each of the varables D and D has the hghest frequency of one, respectvely. So, t s dentfed that model M. (k+) = NP-b-c belongs to case III. Therefore, varable D whch has a = 7-- () weaker absolute correlaton coeffcent wth the = dependent varable Y s then removed from ths model. Thus, the resultng model free from multcollnearty s Therefore, t s shown n Table 8 that the number of M.. parameters left n the selected model M.. s the same Table : Process of removng a multcollnearty source varable from model M Parameters Y D D D Y D D D Frequency Case II Remove 04 Scence Internatonal 4

6 Table 4: Process of removng a multcollnearty source varable from model M. Parameters Y D D D Y D D D Frequency Case Remove II D Table 5: Process of removng a multcollnearty source varable from model M. Parameters Y D D Y D D Frequency Case III Remove D Table 6: Correlaton coeffcent matrx of model M. whch s free from multcollnearty source varable Parameters Y D Y D Table 7: Elmnaton of nsgnfcant varable from model M. Unstandardzed coeffcents Standardzed coeffcent ˆβ Model M. SE Beta tcal p-value Acton Constant G G G Elmnate D G Table 8: Model M.. whch s free from Insgnfcant varable Unstandardzed coeffcents Standardzed coeffcent ˆβ Model M.. SE Beta tcal p-value Acton Constant G End G D G wth the value obtaned from the proposed n Eq.. calculate the number of parameters and demonstrated In lne wth the above dscusson, other researchers ther applcaton on models wth and wthout nteracton also hghlghted the mportance of ths parameters varables. countng. They ranked them (as mportance, As can be seen from prevous secton, t requres sgnfcance or dependency etc.) the parameters of lengthy tme to calculate the number of parameters n a a model based on the magntude of the model, especally for bgger models lke Eq Instead,4,5 coeffcents. of calculatng the number of parameters one by one manually, the equaton establshed n ths study allow CONCLUSION researchers to obtan the number of parameters s an Ths study s new and groundbreakng. It has easer, faster yet accurate way. Besdes, human errors that succeeded n proposng a equaton n countng the are caused by manual countng, (have happened durng number of parameters for each of all possble models model development) can also be mnmsed and avoded.,4 (detals can be found n phase and n ). Ths equaton The proposed equaton also helps to save tremendous helps to countercheck the number of parameters left n amount of tme, where there are analyss nvolvng the selected model whch s free from multcollnearty complex teratons or repeated tasks, especally n and from nsgnfcant varable. It presented a equaton to software development. 04 Scence Internatonal 4

7 REFERENCES 9. Chttawatanarat, K., S. Pruenglampoo,. Ramanathan, R., 00. Introductory Econometrcs V. Trakulhoon, W. Ungpntpong and wth Applcatons. 5th Edn., Harcourt College Publshers, Oho, USA., ISBN-: , J. Patumanond, 0. Development of gender-and age group-specfc equatons for estmatng body Pages: 688. weght from anthropometrc measurement n. Gujarat, D.N. and D.C. Porter, 009. Basc Tha adults. Int. J. Gen. Med., 5: Econometrcs. 5th Edn., McGraw-Hll Companes 0. Zeles, A., 004. Econometrc computng Inc., New York, USA., ISBN-: , wth HC and HAC covarance matrx Pages: 944. estmators. J. Statst. Software, : -7.. Zanodn, H.J., A. Noran and S.J. Yap, 0. An. Ansorge, H.L., S. Adams, A.F. Jawad, D.E. Brk alternatve multcollnearty approach n solvng and L.J. Soslowsky, 0. Mechancal property multple regresson problem. Trends Appled Sc. changes durng neonatal development and healng Res., 6: usng a multple regresson model. J. Bomechan., 4. Yahaya, A.H., N. Abdullah and H.J. Zanodn, 0. 45: Multple regresson models up to frst-order. Emuchay, C.I., S.O. Okeny and J.O. Okeny, 04. nteracton on hydrochemstry propertes. Correlaton between total lymphocyte count, Asan J. Math. Stat., 5: Anastasopoulos, P.C. and F.L. Mannerng, 009. A hemoglobn, hematocrt and CD4 count n HIV note on modelng vehcle accdent frequences wth patents n Ngera. Pak. J. Bol. Sc., 74: random-parameters count models. Accd. Anal.. Furnstahl, R.J. and B.D. Serot, 000. Parameter Prev., 4: countng n relatvstc mean-feld models. Nucl. 6. Mazzocchett, D., A.M. Bert, G. Lucchett, R. Phys. A, 67: Sartn, F. Colle, P. Iudcone and L. Perell, Robnson, P.S., T.W. Ln, A.F. Jawad, R.V. Iozzo Evaluaton of volume and total nucleated cell count and L.J. Soslowsky, 004. Investgatng tendon as cord blood selecton parameters. Am. J. Cln. fasccle structure-functon relatonshps n a Pathol., 8: transgenc-age mouse model usng multple 7. Jaccard, J., 00. Interacton Effects n Logstc regresson models. Ann. Bomed. Eng., Regresson. SAGE Publcatons, London, Pages: 70. : Lnd, D.A., W.G. Marchal and S.A. Wathen, 0. Statstcal Technques n Busness and Economcs. 5th Edn., McGraw-Hll, New York. 5. Festng, M.F., 006. Desgn and statstcal methods n studes usng anmal models of development. ILAR J., 47: Scence Internatonal 4

S1 Note. Basis functions.

S1 Note. Basis functions. S1 Note. Bass functons. Contents Types of bass functons...1 The Fourer bass...2 B-splne bass...3 Power and type I error rates wth dfferent numbers of bass functons...4 Table S1. Smulaton results of type