Synthesizer 1.0. User s Guide. A Varying Coefficient Meta. nalytic Tool. Z. Krizan Employing Microsoft Excel 2007

Transcription

1 Syntheszer 1.0 A Varyng Coeffcent Meta Meta-Analytc nalytc Tool Employng Mcrosoft Excel User s Gude Z. Krzan 009

2 Table of Contents 1. Introducton and Acknowledgments 3. Operatonal Functons 4 a. Correlaton Aggregator 4 b. Dfference Aggregator 6 c. Relablty Generalzator 8 d. Correlaton Contrasts 9 e. Dfference Contrasts Worked Example (***forthcomng***) Computatons 1 5. References 5

3 Introducton and Acknowledgments Thank you for your nterest n Syntheszer 1.0, the latest freeware for meta-analytc computatons usng Mcrosoft Excel 007. Ths s the frst verson of ths no-cost program, so future addtons and mprovements are antcpated. Gven t s based on the Excel platform, t gves you (the user) sgnfcant flexblty n rearrangng ts computatons and functons. If you have any questons about the program, feel free to contact me at zkrzan@astate.edu. I would lke to thank Doug Bonett whose hard work lead to the statstcal models employed n Syntheszer 1.0, and whose advce was ndspensable for developng ths program. Thanks also go to Brad Bushman, Kyle Scherr, and Jeff Mller for ther feedback durng the development process.

4 Operatonal Functons Syntheszer 1.0 s organzed n terms of Analyss Modules, whch are accessble va tabs on the bottom left of the screen. Each Module s a dfferent Excel sheet. 1. Access Analyss Modules va tabs depcted above The frst tab, Man, s the ntroductory screen whch s present when you open the program. It contans a Quck Gude to the basc functons of Syntheszer 1.0. The gude specfes whch functons are avalable n each module, and provdes bref nstructons for how to test for moderatng factors. Correlaton Aggregator (r) The frst Analyss Module s the Correlaton Aggregator, and can be accessed by clckng on the tab denoted [r]. Ths module allows for quanttatve synthess of correlatons from ndependent samples. Pearson ss product-moment, Spearman s rank-order, orpartal correlatons can all be used n ths module. Tetrachorc correlatons or other ndexes based on contngency tables cannot be used n ths module. Requred Input. In orderr to obtan meta-analytc estmates, the user needs to enter only the correlatons and sample szes. Columns for enterng Manuscrpt Numbers, References, Subgroup descrptors, and Sample Identfcaton Numbers are also ncluded. These columns are not nvolved n any computatons; they are there smply for easer reference.. Input nformaton for each study across columns depcted above

5 The only other operaton requred by the user s to paste the formula for computatons of VAR(r) n all rows that contann data (.e., effect szes). Ths s performed n the followng way, and s ndcated on the top of ths row n the actual module. 1. Copy the contents of the H7 cell that contans the value.. In the same column, Paste Specal the contents of ths cell n ALL rows that contan data by frst selectng them, then usng the rght mouse clck, selectng Paste Specal, and ndcatng Formulas from the dalog box. [Ths s necessary n order to compute the correct sum gven the default value s not zero] Optonal Input. To the rght of the VAR(r) column are addtonal columns whch can be used to specfy moderator varables, and nsert codes (0,1, etc.) for dfferent levels of the moderator varable. Steps to test for moderatng effects are descrbed n a later secton. 3. Informaton for moderator categores can be entered n columns depctedd above Output. On the bottom part of the screen, the user can access basc descrptve statstcs of the meta-analytc set, and the crtcal 95% Confdence Interval around the metaas sgnfcant f analytc mean. Although not recommended, one can nterpret the correlaton the Confdence Interval does not nclude zero. Relevant descrptve statstcs are as follows: 1. Total Number of Partcpants [N]; number of ndvduals across all samples. Number of Studes [m]; number of samples (data ponts) ncluded 3. Dsperson of Effect Szes [SD(r)]; standard devaton of effect szes ncluded 4. Meta-analytc Mean [δ]; Estmate of the populaton effect sze 5. Parameter Varance [VAR(ρ)]; Varance of the mean estmate (squared standard error) 4. Descrptve statstcs can be seen on the left, whle the confdence nterval s to the rght

6 Testng Moderaton. In order to test moderaton, the Correlatonal Contrasts [r- Contrasts] module s used (see below). Here are nstructons for how to preparee data for ths module: 1. Specfy the name of the moderatng varable on the top of each moderator column (see above). Insert codes for each level of the moderator. These codes are arbtrary, and any letter or number scheme that the user prefers wll work equally welll (e.g., 1 for studes done wth students, and for studes done wth communty resdents) 3. Select the entre table wth data 4. Use the Excel SORT functon to sort data accordng to values of a gven Moderator column (e.g., Partcpants ) 5. Copy the Excel sheet such that there s a separate sheet for each level of the moderator (e.g., f there are two dfferent types of partcpants, create two addtonal copes) 6. In each coped sheet, DELETE all data but for studes assocated wth one of the moderatng condtons (sortng pror to ths step makes ths step easer) 7. Label each sheet (by clckng on sheet name) accordng to the level of ts moderatng varable for easer reference (e.g., Student partcpants ) 8. Interpret the recalculated meta-analytc statstcs that wll appear on the bottom of the sheet (see above) ), whch wll reflect values for the specfc subgroup 9. To estmate the actual dfference between the moderator groups, use the Correlatonal Contrasts [d Contrasts] module descrbed below Dfference Aggregator (d) The second Analyss Module s the Dfference Aggregator, and can be accessed by clckng on the tab denoted [d]. Ths module allows for quanttatve synthess of standardzed mean dfferences from ndependent samples. Un-standardzed mean dfferences cannot be used n ths module. Requred Input. In orderr to obtan meta-analytc estmates, the user needs to enter the sample szes, means, and standard devatons for the groups beng compared. Alternatvely, the standardzed mean dfferences (Cohen s d) can be entered drectly. Columns for enterng Manuscrpt Numbers, References, Subgroup descrptors, and Sample Identfcaton Numbers are also ncluded for easer reference, but do not requre any nput.. Input nformaton for each study across columns depcted above

7 The only other operaton requred by the user s to paste the formula for computatons of small sample bas correcton (b) n all rows that contan data (.e., effect szes). Ths s performed n the followng way, and s ndcated on the top of ths row n the actual module. 1. Copy the contents of the L6 cell that contans the value.. In the same column, Paste Specal the contents of ths cell n ALL rows that contan data by frst selectng them, then usng the rght mouse clck, selectng Paste Specal, and ndcatng Formulas from the dalog box. [Ths s necessary n order to compute the correct sum gven the default value s not zero] Optonal Input. To the rght are addtonal columns whch can be used to specfy moderator varables, and nsert codes (0,1, etc.) for dfferent levels of the moderator varable. Steps to test for moderatng effects are descrbed n a later secton. 3. Informaton for moderator categores can be entered n columns depctedd above Output. On the bottom part of the screen, the user can access basc descrptve statstcs of the meta-analytc set, and the crtcal 95% Confdence Interval around the metaas sgnfcant f analytc mean. Although not recommended, one can nterpret the dfference the Confdence Interval excludes zero. Relevant descrptve statstcs are as follows: 6. Total Number of Partcpants [N]; number of ndvduals across all samples 7. Number of Studes [m]; number of samples (data ponts) ncluded 8. Dsperson of Effect Szes [SD(r)]; standard devaton of effect szes ncluded 9. Meta-analytc Mean [ρ]; Estmate of the populaton effect sze 10. Parameter Varance [VAR(ρ)]; Varance of the mean estmate (squared standard error) 4. Descrptve statstcs can be seen on the left, whle the confdence nterval s to the rght

8 Testng Moderaton. In order to test moderaton, the Dfference Contrasts [d- Contrasts] module s used (see below). In order to prepare data for testng moderaton, follow the same steps as outlned for correlatons Relablty Generalzator (Alpha) The thrd Analyss Module s the Relablty Generalzator, and can be accessed by clckng on the tab denoted [Alpha]. Ths module allows for quanttatve synthess of nternal consstency estmates (Cronbach s α) from ndependent samples. The nternal consstency estmates mght nvolve any type of observaton s (e.g., test tems, observer judgments), but ther number must reman constant across all the estmates. Requred Input. In orderr to obtan meta-analytc estmates, the user needs to enter only the relablty estmate, the sze of the relevant samples, and the number of test tems. Columns for enterng Manuscrpt Numbers, References, Subgroup descrptors, and Sample Identfcaton Numbers are also ncluded. These columns are not nvolved n any computatons; they are there smply for easer reference.. Input nformaton for each study across columns depcted above The only other operaton requred by the user s to paste the formula for computatons of VAR(α) n all rows that contan data (.e., effect szes). Ths s performed n the followng way, and s also ndcated on the top of ths row n the actual module. 3. Copy the contents of the I6 cell that contans the zero value. 4. In the same column, Paste Specal the contents of ths cell n ALL rows that contan data by frst selectng them, then usng the rght mouse clck, selectng Paste Specal, and ndcatng Formulas from the dalog box. Optonal Input. To the rght of the VAR(r) column are addtonal columns whch can be used to specfy moderator varables, and nsert codes (0,1, etc.) for dfferent levels of the moderator varable. Steps to test for moderatng effects are descrbed n a later secton.

9 3. Informaton for moderator categores can be entered n columns depctedd above Output. On the bottom part of the screen, the user can access basc descrptve statstcs of the meta-analytc set, and the crtcal 95% Confdence Interval around the meta- 1. Number of test observatons [q](e.g., test tems) analytc mean. Relevant descrptve statstcs are as follows: 11. Total Number of Partcpants [N]; number of ndvduals across all samples 13. Number of Studes [m]; number of samples (data ponts) ncluded 14. Dsperson of Effect Szes [SD(α)]; standard devaton of effect szess ncluded 15. Meta-analytc Mean [α]; Estmate of the populaton relablty 16. Parameter Varance [VAR(Alpha)]; Varance of the mean estmate (squared standard error) 17. Remanng calculatons normalze the dstrbuton and correct for small sample bas 4. Descrptve statstcs can be seen on the left, whle the confdence nterval s to the rght Testng Moderaton. In order to test moderaton, the Correlatonal Contrasts [r- Contrasts] module s used (see below). In order to prepare data for testng moderaton, follow the same steps as outlned for correlatons. The followng secton descrbed how to test moderaton for correlatons or nternal consstences. Correlatonal Contrasts (r- Contrasts) The fourth Analyss Module performs Correlatonal Contrasts, and can be accessed by clckng on the tab denoted [r- Contrasts]. Ths module allows for testng dfferences n

10 syntheszed correlatons (r) or nternal consstences (α) from ndependent samples. It requres that 95% Confdence Interval estmates for groups beng compared (e.g., test relablty among men vs. women) already be computed. Requred Input. For each group that s beng compared (A& B), the user only needs to enter the parameter estmate, and the low-bound and hgh-bound estmates based on the 95% Confdence Interval prevously computed. Labels for each group are optonal. Output. On the lower part of the screen the user can access the estmate of the dfference n correlatons (or nternal consstences), together wth assocated 95% Confdence Interval. Although not recommended, one can nterpret the dfference as sgnfcant f the Confdence Interval excludes zero. 4. Confdence Interval values are entered above, whle the confdence nterval around the dfference s below Dfference Contrasts (d- Contrasts) The ffth and fnal Analyss Module performs Dfference Contrasts, and can be accessed by clckng on the tab denoted [d - Contrasts]. Ths module allows for testng dfferences n syntheszed standardzed mean dfferences from ndependent samples. It requres that the means and varances (squared standard errors) from groups beng compared already be computed (see above). Furthermore, the user needs to specfy a contrast weght for each group. Any number of groups can be compared n any combnaton, as long as the weghts sum to zero (e.g., the mean of three groups can be compared to the mean of fve dfferent groups). Requred Input. For each group that s beng compared (A through F), the user only needs to enter the means, varances, and contrast weghts. These wll be avalable upon preparng data for moderaton analyss descrbed earler. Labels for each group are optonal.

11 Output. On the lower part of the screen the user can access the estmate of the dfference n effect estmates across the contrasted groups, together wth assocated 95% Confdence Interval. Although not recommended, one can nterpret the dfference as sgnfcant f the Confdence Interval excludes zero. 5. Group means, varances, and relevant contrasts are entered above, whle the confdence nterval around the dfference s to the lower-rght

12 Worked Example **** Forthcomng *****

13 Computatons Ths secton descrbes all the computatons that are used n Syntheszer 1.0 to derve pont estmates, varances, standard errors, and aggregatons thereof (as developed by Bonett 008, 009, 009b). For further dscusson of these computatons, see Krzan (009). At ths pont, Syntheszer 1.0 requres nput of ether Pearson s product-moment correlatons (r), or Cohen s standardzed mean dfferences (d). Conversons between these and related effect sze ndexes are descrbed n Rosenthal (1991). Correlatons. Mean estmate of populaton correlatons s calculated as follows: m = 1 1 ρ = m ˆρ The varance of ndvdual study estmates are calculated as (1 ˆ ρ ) /( n -3). In order to normalze the samplng dstrbuton of estmates, Fsher s (195) correcton s appled to calculatons of mean estmate and ts varance. The 95% confdence nterval around the mean (Bonett, 008) s calculated as follows: m tanh[tanh -1 1/ ( ρ ) ± z { (1 ˆ α / = ρ ) /( n 3)} / m(1 ρ )] 1 Tests of dfference between populaton values are performed by constructng a 95% confdence nterval around the dfference n the two sets of correlatons, whose upper and lower bounds (Bonett, 009b) are calculated as follows : L = ρ ρ A B {( ρ A LA ) + ( U B ρ B ) } 1/ U = ρ ρ + ρ A B {( U A ρ A ) + ( B LB ) } 1/ Mean Dfferences. Mean estmate of populaton standardzed mean dfferences s calculated as follows: δ = m 1 m = b ˆ 1 δ The b reflects the small sample adjustment orgnally proposed by Hedges (1981), b = 1 3/{4(n 1 + n ) 9}. Varance of each estmate s calculated as follows gven nformaton for the two groups: 4 var( δˆ ) = { ˆ 4 4 δ ( ˆ ˆ 1 / 1 / ) /8 ˆ σ df + σ df σ + ( ˆ σ ˆ ˆ 1 / df 1 + σ / df ) / σ }

14 The 95% confdence nterval around the mean (Bonett, 009) s calculated as follows: m δ ± zα / { m = b ˆ 1 var( δ Lnear contrasts of mean dfferences are performed by frst estmatng the dfference tself (a) and the varance of the contrast estmate (b): )} 1/ (a) m c δˆ m = 1 (b) var( ˆ = c ) 1 δ The sze of the dfference and assocated 95% confdence nterval s calculated as follows: m c δˆ = 1 ± α / m z {var( c δˆ = )} 1/ 1 Internal consstences. Based on Bonett (009), mean estmate of populaton relablty s calculated as follows: m j = 1 1 ρ = m ˆρ. In order to normalze the samplng dstrbuton of estmates, a logarthmc transformaton s appled to calculatons of mean estmate and ts varance. Varance of sample estmates (a) and mean estmate varance (b) s calculated as follows, q reflectng the number of test tems: (a) var( ρˆ j ) q(1 ρˆ j ) /{(q 1)(n )} (b) var( ρ ) m j = var(ρ ˆ 1 j ) Varance of the log transformed estmate s estmated by: var{ln(1 ρ )} var( ρ )/(1 ρ ). Ths transformed value s then used n the computaton of the 95% confdence nterval around the populaton mean (Bonett, 009b), calculated as follows: j 1 exp(ln(1 ρ ) b ± z [var{ln(1 )}] 1/ ) (9) α / ρ In the above formula, b s an approxmate adjustment for the bas ntroduced by logarthmc transformaton b = ln{ n /( n 1), wth n reflectng the harmonc mean sample sze. Test of dfference between populaton relabltes are performed n the same was as for correlatons (see above and Bonett, 008). m

15 References Bonett, D.G (008). Meta-analytc nterval estmaton for Pearson correlatons. Psychologcal Methods, 13, Bonett, D. G. (009a). Meta-analytc nterval estmaton for standardzed and unstandardzed mean dfferences. Psychologcal Methods. Bonett, D. G. (009b). Meta-analytc confdence nterval for alpha relablty coeffcents. Manuscrpt submtted for publcaton. Hedges, L. V. (1981). Dstrbuton theory for Glass s estmator of effect sze and related estmators. Journal of Educatonal Statstcs, 6, Krzan, Z. (009). Syntheszer 1.0: A varyng coeffcent meta-analytc tool. Manuscrpt submtted for publcaton. Rosenthal, R. (1991). Meta-analytc Procedures for Socal Research. Newbury Park, CA: Sage.