SEM 1: Confirmatory Factor Analysis

Transcription

1 SEM 1: Confirmatory Factor Analysis Week 3 - Measurement invariance and ordinal data Sacha Epskamp

2 General factor analysis framework: in which: y i = Λη i + ε i y N(0, Σ) η N(0, Ψ) ε N(0, Θ), y i is a p-length vector of item responses η i an m-length vector of latent variables ε i an p-length vector of residuals Λ a p m matrix of factor loadings Ψ an m m symmetric variance covariance matrix (assume always all latent variables are correlated) Θ is a p p symmetric variance covariance matrix, mostly diagonal (unless you explicitly expect violations of local independence)

3 The general framework: y i = Λη i + ε i y N(0, Σ) η N(0, Ψ) ε N(0, Θ), Allows you to derive the model-implied variance covariance matrix: Σ = ΛΨΛ + Θ

4 Two main rules: Identification Because the unit of the latent variable is unknown, we need to scale the latent variable by fixing its variance to 1 or fixing one (usually the first) factor loading to 1. We need at least as many observations (sample variances and covariances) as the number or parameters; we require non-negative degrees of freedom (DF) DF = a b a: number of observations: a = p(p + 1)/2 variances and covariances. b: number of parameters we need to estimate (do not count parameters we fixed for scaling) In general, we need 3 indicators for a single latent variable model, or 2 per factor for models with multiple (correlated) latent variables.

5 Lavaan # Install the package: install.packages("lavaan") # Load the package: library("lavaan") # Read data into R: Data <- read.csv("holzingerswineford1939.csv")

6 # cfa() model: Model <- ' visual =~ x1 + x2 + x3 textual =~ x4 + x5 + x6 speed =~ x7 + x8 + x9 '

7 # Fit in lavaan: fit <- cfa(model, Data) # Assess fit: fit ## lavaan ( ) converged normally after 35 iterations ## ## Number of observations 301 ## ## Estimator ML ## Minimum Function Test Statistic ## Degrees of freedom 24 ## P-value (Chi-square) 0.000

8 Fitting CFA models lavaan (R), Onyx and Jasp Testing for exact fit χ 2 test Assessing close fit RMSEA (below 0.5 to 0.8) SRMR (below 0.5) CFI, RNI, NFI, TLI, RFI, IFI (above 0.90 to 0.95) (A) GFI (above 0.90) Model comparison Likelihood ratio test Information criteria Modification indices

9 Lagrange Multiplier (LM) Tests mod <- modindices(fit) library("dplyr") mod %>% arrange(-mi) %>% head(10) ## lhs op rhs mi epc sepc.lv sepc.all sepc ## 1 visual =~ x ## 2 x7 ~~ x ## 3 visual =~ x ## 4 x8 ~~ x ## 5 textual =~ x ## 6 x2 ~~ x ## 7 textual =~ x ## 8 x2 ~~ x ## 9 x3 ~~ x ## 10 visual =~ x

10 # Adjusted model: Model <- ' visual =~ x1 + x2 + x3 + x9 textual =~ x4 + x5 + x6 speed =~ x7 + x8 + x9 x3 ~~ x5 '

11 # Fit in lavaan: fit <- cfa(model, Data) # Assess fit: fitmeasures(fit, c("rmsea","cfi","tli","rni","rfi","ifi","srmr","gfi")) ## rmsea cfi tli rni rfi ifi srmr gfi ##

12 Today Mean structure Multi-group CFA Measurement invariance Ordinal data

13 Including the mean structure: y i = τ + Λη i + ε i y N(µ, Σ) η N(α, Ψ) ε N(0, Θ), Allows you to derive the model-implied variance covariance matrix and means vector: Σ = ΛΨΛ + Θ µ = τ + Λα µ should resemble sample means ȳy as closely possible

14 Σ = ΛΨΛ + Θ µ = τ + Λα τ can cancel α out, hence we need to identify α = 0 Number of parameters: p(p + 1)/2 variances and covariances and p means! Number of parameters: p intercepts in τ p more observations, and p more parameters. This is why we normally ignore means!

15

16 Measurement Invariance f (y η, s) = f (y η) The distribution of y is independent on class membership s after knowing the latent trait η. Mellenbergh, G. J. (1989). Item bias and item response theory. International journal of educational research, 13(2),

17 Measurement invariance holds:

18 Measurement invariance is violated:

19 Measurement invariance is violated:

20 Multi-group model

21 Two models, for group 1: Σ 1 = Λ 1 Ψ 1 Λ 1 + Θ 1 µ 1 = τ 1 + Λ 1 α 1 With observed variance covariance matrix S 1 and observed means ȳȳȳ 1. For group 2: Σ 2 = Λ 2 Ψ 2 Λ 2 + Θ 2 µ 2 = τ 2 + Λ 2 α 2 With observed variance covariance matrix S 2 and observed means ȳȳȳ 2. Twice as many observations and parameters.

22 Steps to assess measurement invariance: Configural invariance: Is the configuration of the model the same? Weak Invariance: Are factor loadings the same? Strong Invariance: Are the intercepts the same? Strict Invariance: Are the residual variances the same?

23 Wicherts, J. M., & Dolan, C. V. (2010). Measurement invariance in confirmatory factor analysis: An illustration using IQ test performance of minorities. Educational Measurement: Issues and Practice, 29(3),

24 Wicherts, J. M., & Dolan, C. V. (2010). Measurement invariance in confirmatory factor analysis: An illustration using IQ test performance of minorities. Educational Measurement: Issues and Practice, 29(3),

25 Configural invariance: Does the same model fit in both groups?

26 Configural invariance For group 1: Σ 1 = Λ 1 Ψ 1 Λ 1 + Θ 1 µ 1 = τ 1 For group 2: Σ 2 = Λ 2 Ψ 2 Λ 2 + Θ 2 µ 2 = τ 2 (latent variable means constrained to be zero)

27 Model <- ' visual =~ x1 + x2 + x3 + x9 textual =~ x4 + x5 + x6 speed =~ x7 + x8 + x9 x3 ~~ x5 ' fit_configural <- cfa(model, Data, group = "school") fitmeasures(fit_configural, c("rmsea","cfi","tli","rni","rfi","ifi","srmr","gfi")) ## rmsea cfi tli rni rfi ifi srmr gfi ##

28 library("semplot") layout(t(1:2)) sempaths(fit_configural, "mod", "est", ask=false, layout = "tree", levels = c(1,2,4,5), edge.color = "black", reorder = FALSE, manifests = paste0("x",1:9), mar = c(2,2,4,2)) vsl txt spd vsl txt spd x1 x2 x3 x4 x5 x6 x7 x8 x9 x1 x2 x3 x4 x5 x6 x7 x8 x

29 Weak Invariance: Are factor loadings the same?

30 Weak Invariance Λ 1 = Λ 2 = Λ. For group 1: Σ 1 = ΛΨ 1 Λ + Θ 1 µ 1 = τ 1 For group 2: Σ 2 = ΛΨ 2 Λ + Θ 2 µ 2 = τ 2

31 Model <- ' visual =~ c(l1,l1)*x1 + c(l2,l2)*x2 + c(l3,l3)*x3 + c(l10,l10) * x9 textual =~ c(l4,l4)*x4 + c(l5,l5)*x5 + c(l6,l6)*x6 speed =~ c(l7,l7)*x7 + c(l8,l8)*x8 + c(l9,l9)*x9 x3 ~~ x5 ' fit_weak <- cfa(model, Data, group = "school") fit_weak ## lavaan ( ) converged normally after 41 iterations ## ## Number of observations per group ## Pasteur 156 ## Grant-White 145 ## ## Estimator ML ## Minimum Function Test Statistic ## Degrees of freedom 51 ## P-value (Chi-square) ##

32 Model <- ' visual =~ x1 + x2 + x3 + x9 textual =~ x4 + x5 + x6 speed =~ x7 + x8 + x9 x3 ~~ x5 ' fit_weak <- cfa(model, Data, group = "school", group.equal = "loadings") fit_weak ## lavaan ( ) converged normally after 41 iterations ## ## Number of observations per group ## Pasteur 156 ## Grant-White 145 ## ## Estimator ML ## Minimum Function Test Statistic ## Degrees of freedom 51 ## P-value (Chi-square) ##

33 layout(t(1:2)) sempaths(fit_weak, "mod", "est", ask=false, layout = "tree", levels = c(1,2,4,5), edge.color = "black", reorder = FALSE, manifests = paste0("x",1:9), mar = c(2,2,4,2)) vsl txt spd vsl txt spd x1 x2 x3 x4 x5 x6 x7 x8 x9 x1 x2 x3 x4 x5 x6 x7 x8 x

34 fitmeasures(fit_weak, c("rmsea","cfi","tli","rni","rfi","ifi","srmr","gfi")) ## rmsea cfi tli rni rfi ifi srmr gfi ## anova(fit_configural, fit_weak) ## Chi Square Difference Test ## ## Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq) ## fit_configural ## fit_weak (7 DF difference: 7 factor loadings)

35 Strong Invariance: Are the intercepts the same?

36 Strong Invariance τ 1 = τ 2 = τ. For group 1: Σ 1 = ΛΨ 1 Λ + Θ 1 µ 1 = τ For group 2: Σ 2 = ΛΨ 2 Λ + Θ 2 µ 2 = τ + Λα 2 α 2 is now identified!

37 Model <- ' visual =~ x1 + x2 + x3 + x9 textual =~ x4 + x5 + x6 speed =~ x7 + x8 + x9 x3 ~~ x5 ' fit_strong <- cfa(model, Data, group = "school", group.equal = c("loadings","intercepts")) fit_strong ## lavaan ( ) converged normally after 56 iterations ## ## Number of observations per group ## Pasteur 156 ## Grant-White 145 ## ## Estimator ML ## Minimum Function Test Statistic ## Degrees of freedom 57 ## P-value (Chi-square) ##

38 layout(t(1:2)) sempaths(fit_strong, "mod", "est", ask=false, layout = "tree", levels = c(1.25,2,4,5), edge.color = "black", reorder = FALSE, manifests = paste0("x",1:9), mar = c(2,2,4,2)) vsl txt spd vsl txt spd x1 x2 x3 x4 x5 x6 x7 x8 x9 x1 x2 x3 x4 x5 x6 x7 x8 x

39 fitmeasures(fit_strong, c("rmsea","cfi","tli","rni","rfi","ifi","srmr","gfi")) ## rmsea cfi tli rni rfi ifi srmr gfi ## anova(fit_configural, fit_weak, fit_strong) ## Chi Square Difference Test ## ## Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq) ## fit_configural ## fit_weak ## fit_strong e-07 ## --- ## Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 (6 DF difference: 9 intercepts, but +3 latent variable means)

40 Where is the misfit? # Group 1: lavinspect(fit_strong, "mu")[[1]] - lavinspect(fit_strong, "sampstat")[[1]]$mean ## x1 x2 x3 x9 x4 x5 x6 x7 x8 ## # Group 2: lavinspect(fit_strong, "mu")[[2]] - lavinspect(fit_strong, "sampstat")[[2]]$mean ## x1 x2 x3 x9 x4 x5 x6 x7 x8 ##

41 Free two intercepts: Model <- ' visual =~ x1 + x2 + x3 + x9 textual =~ x4 + x5 + x6 speed =~ x7 + x8 + x9 x3 ~~ x5 x3 ~ c(t31,t32) * 1 x7 ~ c(t71,t72) * 1 ' fit_strong_mod <- cfa(model, Data, group = "school", group.equal = c("loadings","intercepts"))

42 fitmeasures(fit_strong_mod, c("rmsea","cfi","tli","rni","rfi","ifi","srmr","gfi")) ## rmsea cfi tli rni rfi ifi srmr gfi ## anova(fit_configural, fit_weak, fit_strong, fit_strong_mod) ## Chi Square Difference Test ## ## Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq) ## fit_configural ## fit_weak ## fit_strong_mod ## fit_strong e-08 ## --- ## Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

43 Free two intercepts: Model <- ' visual =~ x1 + x2 + x3 + x9 textual =~ x4 + x5 + x6 speed =~ x7 + x8 + x9 x3 ~~ x5 ' fit_strong_mod <- cfa(model, Data, group = "school", group.equal = c("loadings","intercepts"), group.partial = c("x3 ~ 1","x7 ~ 1"))

44 fitmeasures(fit_strong_mod, c("rmsea","cfi","tli","rni","rfi","ifi","srmr","gfi")) ## rmsea cfi tli rni rfi ifi srmr gfi ## anova(fit_configural, fit_weak, fit_strong, fit_strong_mod) ## Chi Square Difference Test ## ## Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq) ## fit_configural ## fit_weak ## fit_strong_mod ## fit_strong e-08 ## --- ## Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

45 Strict Invariance: Are the residual variances the same?

46 Strict Invariance Θ 1 = Θ 2 = Θ. For group 1: Σ 1 = ΛΨ 1 Λ + Θ µ 1 = τ For group 2: Σ 2 = ΛΨ 2 Λ + Θ µ 2 = τ + Λα 2

47 Model <- ' visual =~ x1 + x2 + x3 + x9 textual =~ x4 + x5 + x6 speed =~ x7 + x8 + x9 x3 ~~ x5 ' fit_strict <- cfa(model, Data, group = "school", group.equal = c("loadings","intercepts","residuals", "residual.covariances"), group.partial = c("x3 ~ 1","x7 ~ 1")) fit_strict ## lavaan ( ) converged normally after 61 iterations ## ## Number of observations per group ## Pasteur 156 ## Grant-White 145 ## ## Estimator ML ## Minimum Function Test Statistic ## Degrees of freedom 65

48 layout(t(1:2)) sempaths(fit_strong, "mod", "est", ask=false, layout = "tree", levels = c(1,2,4,5), edge.color = "black", reorder = FALSE, manifests = paste0("x",1:9), mar = c(2,2,4,2)) vsl txt spd vsl txt spd x1 x2 x3 x4 x5 x6 x7 x8 x9 x1 x2 x3 x4 x5 x6 x7 x8 x

49 fitmeasures(fit_strict, c("rmsea","cfi","tli","rni","rfi","ifi","srmr","gfi")) ## rmsea cfi tli rni rfi ifi srmr gfi ## anova(fit_configural, fit_weak, fit_strong_mod, fit_strict) ## Chi Square Difference Test ## ## Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq) ## fit_configural ## fit_weak ## fit_strong_mod ## fit_strict ## --- ## Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Conclusion: Measurement variance holds adequately except for intercepts of x3 (Lozenges) and x7 (Speeded addition).

50 What about Ψ? If the goal is measurement, these do not matter (factor variances could easily differ per group) If the goal is explanatory, these (or structural relations between latents) could be constrained in a final step Group 1 Group 2 η

51 JASP DEMONSTRATION

52 In Onyx, you need to load separate datasets and copy the entire model. Very lengthy process that is much faster in Jasp or Lavaan..

53 Ordinal data If data is ordinal and consists of only a few levels of measurement data cannot be assumed normal Roughly less than five categories. Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological methods, 17(3), In this case threshold models should be used Then, it is assumed that underlying the response is a latent item that is normally distributed The covariance between this latent items and other such latent items or other continuous items can be estimated Polychoric correlation if both variables are ordinal Polyserial correlation if one item is ordinal and the other is continuous

54 I see myself as someone who is talkative

55

56

57 set.seed(1) # Setup: samplesize < cor <- 0.5 thresh1 <- c(-2,0,2) thresh2 <- c(-1,0.5,1.6) # Generate data: library("mvtnorm") cormat <- matrix(c(1,0.5,0.5,1),2,2) Data <- as.data.frame(rmvnorm(samplesize, sigma = cormat)) # Make catagorical: Data[,1] <- as.numeric(cut(data[,1],breaks = c(-inf,thresh1,inf))) Data[,2] <- as.numeric(cut(data[,2],breaks = c(-inf,thresh2,inf)))

58 # Pearson correlation: cor(data[,1], Data[,2]) ## [1] # Polychoric correlation: library("lavaan") DataOrdered <- Data DataOrdered[,1] <- ordered(data[,1]) DataOrdered[,2] <- ordered(data[,2]) lavcor(dataordered) ## V1 V2 ## V ## V

59 Polychoric correlations Lavaan will automatically treat variables that are made ordered factors via ordered() as ordinal variables and will include thresholds Alternatively, the operator can be used to define thresholds Polychoric and polyserial correlations relax the assumption of normality. However, they can sometimes go wrong! The crosstable should not have zero elements! When testing measurement invariance, now the thresholds need to be equated instead of intercepts

60 No thresholds: table(data) ## V2 ## V ## ## ## ## Zeroes.. So a bit dangerous!

61 No thresholds: Model <- ' a ~~ b ' names(data) <- c("a","b") fit <- cfa(model, Data) parameterestimates(fit) ## lhs op rhs est se z pvalue ci.lower ci.upper ## 1 a ~~ b ## 2 a ~~ a ## 3 b ~~ b

62 Thresholds: Model <- ' a ~~ b a t1 + t2 + t3 b t1 + t2 + t3 ' names(data) <- c("a","b") fit <- cfa(model, Data) parameterestimates(fit) ## lhs op rhs est se z pvalue ci.lower ci.upper ## 1 a ~~ b ## 2 a t ## 3 a t ## 4 a t ## 5 b t ## 6 b t ## 7 b t ## 8 a ~~ a NA NA ## 9 b ~~ b NA NA ## 10 a ~*~ a NA NA ## 11 b ~*~ b NA NA

63 Or use data with ordered columns: Model <- ' a ~~ b ' names(dataordered) <- c("a","b") fit <- cfa(model, DataOrdered) parameterestimates(fit) ## lhs op rhs est se z pvalue ci.lower ci.upper ## 1 a ~~ b ## 2 a t ## 3 a t ## 4 a t ## 5 b t ## 6 b t ## 7 b t ## 8 a ~~ a NA NA ## 9 b ~~ b NA NA ## 10 a ~*~ a NA NA ## 11 b ~*~ b NA NA ## 12 a ~ NA NA ## 13 b ~ NA NA

64 Sample Size How big is big enough? n : q ratio should be high Theory: to efficiently estimate lots of parameters, a larger sample is needed (5-10 per parameter) There s very little evidence that it matters (Jackson, 2003) This ratio is less important than absolute sample size n 200 people This is median SEM sample size (Shah & Goldstein, 2006) Appropriate for an average model with ML estimation Other recommendations: people minimum Use larger n if: Assumptions are violated (e.g., data are nonnormal) Model is complex (e.g., latent interactions, multilevel structure) Indicators have low reliability (factor loadings are low)

65 Big enough for what? Big enough that S is a precise estimate of Σ No estimation problems (model converges) Parameter estimates have small confidence intervals Power to detect model misspecification Chi-square test statistic has sufficient power Fit statistics are accurate

66 Power to detect non-zero parameters G-power cannot help you here: there are too many factors! To estimate power, you need to know (or estimate) the model, and all parameter values Simulation Method for Estimating Power 1. Specify a population model with all parameter values 2. Draw a large number of sample datasets of size n from this hypothetical population (e.g., 1000) 3. Fit the model to each dataset and record whether the parameter value you care about is significant 4. Count the proportion of significant parameter estimates out of 1000 datasets = power

67 Power to detect non-zero parameters G-power cannot help you here: there are too many factors! To estimate power, you need to know (or estimate) the model, and all parameter values Simulation Method for Estimating Power 1. Specify a population model with all parameter values 2. Draw a large number of sample datasets of size n from this hypothetical population (e.g., 1000) simulatedata in lavaan 3. Fit the model to each dataset and record whether the parameter value you care about is significant 4. Count the proportion of significant parameter estimates out of 1000 datasets = power

68 Power to Detect Misspecification Again, Simulation: 1. Specify a population model 2. Draw a large number of sample datasets of size n from this hypothetical population (e.g., 1000) 3. Fit a misspecified model to each dataset and record whether the chi-square test statistic is significant 4. Count the proportion of significant test statistics out of 1000 datasets = power

69 Power for Test of (Not-)Close Fit RMSEA estimates a population value Its sampling distribution has been worked out So we can put a confidence interval around it This confidence interval allows us to ask whether RMSEA is significantly different from a specified value If the population model fit is NOT CLOSE, what is power to reject H 0 by the test of close fit? If the population model fit is CLOSE, what is power to reject H 0 by the test of not-close fit? Method described in MacCallum et al. (1996) is implemented in online calculators: Power and minimum sample size for RMSEA: Power curves for RMSEA: See also findrmseasamplesize in semtools

70 Sample size required to reject RMSEA < 0.05 is the true RMSEA = 0.8 and DF = 20: library("semtools") findrmseasamplesize(rmsea0=.05, rmseaa=.08, df=20, power=0.80) ## [1] 434 Sample size required to reject RMSEA > 0.05 is the true RMSEA = 0.1 and DF = 20: findrmseasamplesize(rmsea0=.05, rmseaa=.01, df=20, power=0.80) ## [1] 474

71 Conclusion Means can be added to the CFA model In multiple-group CFA, a CFA model is fitted to several groups Measurement invariance can be assessed stepwise: Configural invariance: Is the configuration of the model the same? Weak Invariance: Are factor loadings the same? Strong Invariance: Are the intercepts the same? Strict Invariance: Are the residual variances the same? When data are ordinal, polychoric and polyserial correlations can be computed Sample size requirements are complicated, but power can be computed for RMSEA test of (non)close fit