Reliability Coefficients


 Edith Long
 4 days ago
 Views:
Transcription
1 Reliability Coefficients Introductory notes That data used for these computations is the pretreatment scores of all subjects. There are three items in the SIAS (5, 9 and 11) that require reversescoring. Those items are stored in the data frame in reversecoded format. In other words, if a subject responded to one of those items with a score of 0, it has been coded as a 4; if the client responded with a score of 1 it has been coded as a 3, and so on. The following code computes categorical McDonald s omega using the MBESS::ci.reliability() function, classical Cronbach s alpha and the mean interitem correlation using psych::alpha(). In general, we would expect McDonald s omega values to be higher than the alpha values, as (i) they are based on polychoric correlations, rather than Pearson correlations, and (ii) because it is well known in the psychometric literature that alpha tends to underestimate reliability for congeneric measurement models. # Load libraries library(dplyr) Attaching package: 'dplyr' The following objects are masked from 'package:stats': filter, lag The following objects are masked from 'package:base': intersect, setdiff, setequal, union library(psych) library(mbess) Attaching package: 'MBESS' The following object is masked from 'package:psych': cor2cov library(lavaan) This is lavaan lavaan is BETA software! Please report any bugs. Attaching package: 'lavaan' 1
2 The following object is masked from 'package:mbess': cor2cov The following object is masked from 'package:psych': cor2cov # Load datafile load("dat.rda") # Setup dummy matrix for storing store reliabilty coefficients x < matrix(na, nrow = 12, ncol = 3) rownames(x) < c("sias20", "SIAS19", "SIAS17", "SIAS6", "SIPSINT", "SPS", "SPS6", "SIPSFOE", "SIPSFAA", "SIPSAGG", "Peters", "SIPSTot") colnames(x) < c("omega", "alpha", "mean_r") # Function to compute omega, alpha, and mean interitem correlation rel_stats < function(data, items){ df < data[, items] x < c(mbess::ci.reliability(data = df, type = "categorical", interval.type = "none")$est, psych::alpha(df)[[1]][c(1,4)] %>% unlist() ) names(x) < c("omega", "alpha", "mean_r") x } # Compute Omega, alpha and meaninteritem correlation # Extract SIAS items at pretreatment df < dat %>% select(sias_0_1:sias_0_20) # Compute stats for SIAS measures x[1, ] < rel_stats(df, 1:20) # sias20 x[2, ] < rel_stats(df, c(1:4, 6:20)) # sias19 x[3, ] < rel_stats(df, c(1:4, 6:8, 10, 12:20)) # sias17 x[4, ] < rel_stats(df, c(2, 4, 6, 8, 10, 13)) # sias6 x[5, ] < rel_stats(df, c(7, 10, 15, 16, 19)) # sipsint # Extract SPS items at pretreatment df2 < dat %>% select(sps_0_1:sps_0_20) 2
3 # Compute stats for SPS measures x[6, ] < rel_stats(df2, 1:20) # sps20 x[7, ] < rel_stats(df2, c(4, 7, 8, 15:17)) # sps6 x[8, ] < rel_stats(df2, c(4, 6, 8, 13, 16, 17)) # sipsfoe x[9, ] < rel_stats(df2, c(12, 14, 15)) # sipsfaa x[10, ] < rel_stats(df2, c(4, 6, 8, 12:17)) # sipsagg # Peters total (SIAS6 + SPS6) df3 < cbind(df[,c(2, 4, 6, 8, 10, 13)], df2[, c(4, 7, 8, 15:17)]) x[11, ] < rel_stats(df3, 1:12) # SIPS total df4 < cbind(df[,c(7, 10, 15, 16, 19)], df2[, c(4, 6, 8, 12:17)]) x[12, ] < rel_stats(df4, 1:14) # sips total # Display coefficients round(x, 2) omega alpha mean_r SIAS SIAS SIAS SIAS SIPSINT SPS SPS SIPSFOE SIPSFAA SIPSAGG Peters SIPSTot Note that in the above code, separate factor models were specified for each SIPS subscale. An alternate way of computing omega for the SIPS is to define the full two or threefactor model, and use those solutions to compute omega. library(semtools) # This is semtools All users of R (or SEM) are invited to submit functions or ideas for functions. # 3
4 Attaching package: 'semtools' The following object is masked from 'package:psych': skew # Rearrange SIPS items so that items 15 = interaction scale, 611 are foe and faa df5 < cbind(df[,c(7, 10, 15, 16, 19)], df2[, c(4, 6, 8, 13, 16, 17, 12, 14, 15)]) names(df5) < paste0("v", 1:14) # SIPS 3 factor model m1 < ' INT =~ v1 + v2 + v3 + v4 + v5 FOE =~ v6 + v7 + v8 + v9 +v10 + v11 FAA =~ v12 + v13 + v14 ' # Omega = Raykov's omega, omega2 = bentler's omega, omega3 = McDonald's omega # Note that alpha is model based on polychoric correlations and thus will differ # from traditional alpha (in the above table) computed using Pearson correlations fit_m1 < cfa(m1, data = df5, estimator = "WLSMV", std.lv = TRUE, orthogonal = FALSE, missing = "pairwise") round(semtools::reliability(fit_m1),2) INT FOE FAA total alpha omega omega omega avevar The code below compute omega for the SIPS twofactor model, where the two performance anxiety subscales (FOE and FAA) are aggregated. m2 < ' INT =~ v1 + v2 + v3 + v4 + v5 AGG =~ v6 + v7 + v8 + v9 +v10 + v11 + v12 + v13 + v14 ' fit_m2 < cfa(m2, data = df5, estimator = "WLSMV", std.lv = TRUE, orthogonal = FALSE, missing = "pairwise") round(semtools::reliability(fit_m2),2) INT AGG total 4
5 alpha omega omega omega avevar SIAS6 correlation matrix The reliability coefficients for the SIAS6 is notably lower than for the other scales. We can probe the reason for this by examining the interitem correlations which should be quite low (the mean interitem correlation is.27). cor(df[, c(2, 4, 6, 8, 10, 13)]) %>% round(2) sias_0_2 sias_0_4 sias_0_6 sias_0_8 sias_0_10 sias_0_13 sias_0_ NA NA sias_0_4 NA 1 NA NA NA NA sias_0_ NA NA sias_0_ NA NA sias_0_ NA NA sias_0_13 NA NA NA NA NA 1 5