/Users/astacbf/Desktop/Assessing smartpls (engelsk).docx 1/8 Assessing smartpls Everything taken from (Hair, Hult et al. 017) but some formulas taken elswere or created by Erik Mønness. Run PLS algorithm, save picture and EXCEL diagnostics. Run Bootstrap to get significances and confidence intervals. Save EXCEL (as separate file) Run Blindfolding Run Important-Performant map. Save pictures and EXCEL every time if needed. Innhold Assessing smartpls... 1 Stage 5a: Reflective Measurement Models... Internal consistency... Converget Validity... Discriminant validity... 3 Rules of Thumb for evaluation Reflective Measurement Models... 4 Stage 5b: Formative Measurement Models... 5 Collinearity issues... 5 Significance and relevance... 6 Rules of Thumb for the Evaluation of Formative Measurement Indicators... 6 Stage 6: Evaluation of the Structural Model... 7 Rules of Thumb for Structural Model Evaluation... 7 Importance-Performance map.... 8 References... 8
/Users/astacbf/Desktop/Assessing smartpls (engelsk).docx /8 P106 Systematic Evaluation of PLS-SEM Results Stage 5: Evaluation of the Measurement Models Stage 5a: Reflective Measurement Models Internal consistency (Cronbach's alpha, composite reliability) Convergent validity (indicator reliabiiity, average variance extracted) Discriminant validity Stage 6: Evaluation of the Structural Model Coefficients of determination (R ) Predictive relevance (Q ) Size and significance of path coefficients f effect sizes q effect sizes Stage 5b: Formative Measurement Models Convergent validity Collinearity between indicators Significance and relevance of outer weights Stage 5a: Reflective Measurement Models Internal consistency M æ ö si M ç å i 1 Cronbach a = ç 1- = M -1 ssum ç è ø composite reliability ö l ç i i c = è ø æ ö ç åli + å i i r è æ ø å var ( e ) Converget Validity Each loading should be significant and >=0.708 Thus variance explained >=0.5. See 4.4 Average variance extracted AVE M æ ö ç åli i= 1 AVE = ç If standardized. M ç è ø i «Sum of variances divided by variance of sum». If total independence, these are equal, making Cronbach alpha=0. NB Involved indicators must have non-negative correlations. Rescale if needed! l i is outer loading. Var(e i)= 1-l i 0.6-0.7: ok in exploratory analysis. How much variation in the measures are in the construct. Should be >=0.5
/Users/astacbf/Desktop/Assessing smartpls (engelsk).docx 3/8 Discriminant validity Is a construct distinct from others? Cross-loadings Fornell-Larcker Criterion Heterotrait-Monotrait ratio. HTMT. An estimate of the correlation between two constructs. Monotrait= correlations between indicators of same construct. Heterotrait= correlations between indicators from the two separate indicator groups. An indicator s outer loading with its construct should be greater than any of its cross-loadings (correlations) with other constructs. AVE( y ) ³ corr( y, y ) i i j The construct y i has more in common with its indicators than with other constructs. Considered best of the three measures
/Users/astacbf/Desktop/Assessing smartpls (engelsk).docx 4/8 IF HTMT> 0.9 the constructs are not distinct. If bootstrap interval includes 1 they are not distinct. Rules of Thumb for evaluation Reflective Measurement Models Internal consistency reliability: composite reliability should be higher than 0.70 (in exploratory research, 0.60 to 0.70 is considered acceptable). Consider Cronbach's alpha as the lower bound and composite reliability as the upper bound of internal consistency reliability. Indicator reliability: the indicator's outer loadings should be higher than 0.70. Indicators with outer loadings between 0.40 and 0.70 should be considered for removal only if the deletion leads to an increase in composite reliability and AVE above the suggested threshold value. Convergent validity: the AVE should be higher than 0.50. Discriminant validity: Use the HTMT criterion to assess discriminant validity in PLS-SEM. The confidence interval of the HTMT statistic should not include the value 1 for all combinations of constructs. According to the traditional discriminant validity assessment methods, an indicator's outer loadings on a construct should be higher than all its cross-loadings with other constructs. Furthermore, the square root of the AVE of each construct should be higher than its highest correlation with any other construct (Fornell-Larcker criterion)
/Users/astacbf/Desktop/Assessing smartpls (engelsk).docx 5/8 Stage 5b: Formative Measurement Models Formative Measurement Models Assessment Procedure 1 Assess convergent validity of formative measurement models Assess formative measurement models for collinearity issues 3 Assess the significance and relevance of the formative indicators Collinearity issues Variance Inflation Factor VIF =1/TOL is the correlation between one indicator TOLerance = - 1 Rx 1 R x 1 and the other indicators in the same construct. Measures collinearity. TOL<0. or VIF>5 indicate a problem If totally uncorrelated, max outer weight of an indicator is 1 n where n=number of indicators. Thus, when n large, some will appear insignificant. à0.7, 5à0.44, 10à0.31, 0à0.
/Users/astacbf/Desktop/Assessing smartpls (engelsk).docx 6/8 Significance and relevance Rules of Thumb for the Evaluation of Formative Measurement Indicators Assess the formative construct's convergent validity by examining its correlation with an alternative measure of the construct, using reflective measures or a global single item (redundancy analysis). The correlation between the constructs should be 0.70 or higher. Collinearity of indicators: Each indicator's VIF value should be lower than 5. Otherwise, consider eliminating indicators, merging indicators into a single index, or creating higherorder constructs to treat collinearity problems. Examine each indicator's outer weight (relative importance) and outer loading (absolute importance) and use bootstrapping to assess their significance. When an indicator's weight is significant, there is empirical support to retain the indicator. When an indicator's weight is not significant but the corresponding item loading is relatively high (i.e., >=0.50), or statistically significant, the indicator should generally be retained. If the outer weight is non-significant and the outer loading relatively low (i.e., <0.5), you should strongly consider to remove the formative indicator from the model.
/Users/astacbf/Desktop/Assessing smartpls (engelsk).docx 7/8 Stage 6: Evaluation of the Structural Model Structural Model Assessment Procedure Assess structural model VIF values of inner model (among constructs). VIF>5 indicate problems for collinearity issues Assess the significance Statistical significance vs. relevance of significant relations. and relevance of the structural model relationships Assess the level of R n -1 R How much is explained by explaining adj = 1-( 1-R ) n - k -1 constructs. Assess the f effect size Does contribute? y Excluded RIncluded - RExcluded f = 1- RIncluded Thumb rule: small=0.0, medium=0.15, large=0.35 Assess the predictive relevance Q Assess the q effect size Blindfolding procedure. Blindfolding procedure QIncluded - QExcluded q = 1- Q Included Rules of Thumb for Structural Model Evaluation Examine each set of predictors in the structural model for collinearity. Each predictor construct's tolerance (VIF) value should be higher than 0.0 (lower than 5). Otherwise, consider eliminating constructs, merging predictors into a single construct, or creating higher-order constructs to treat collinearity problems. Use bootstrapping to assess the significance of path coefficients. The minimum number of bootstrap samples must be at least as large as the number of valid observations but should be 5,000. Critical t values for a two-tailed test are 1.65 (significance level = 10%), 1.96 (significance level = 5%), and.57 (significance level = 1 %). Alternatively, examine the p value, which should be lower than 0.10 (significance level= 10%), 0.05 (significance level = 5%), or 0.01 (significance level = 1 %). In applications, you should usually assume a 5% significance level. Bootstrap confidence intervals provide additional information on the stability of path coefficient estimates. Use the percentile method for constructing confidence intervals. When models are not complex (i.e., fewer than four constructs) and sample size is small, use double bootstrapping. However, the running time can be extensive. PLS-SEM aims at maximizing the R values of the endogenous latent variable(s) in the path model. While the exact interpretation of the R value depends on the particular model and research discipline, in general R values of 0.75, 0.50, or 0.5 for the endogenous construct can be described as respectively substantial, moderate, and weak. Use the. when comparing models with different exogenous constructs and/or R adj different numbers of observations..
/Users/astacbf/Desktop/Assessing smartpls (engelsk).docx 8/8 The effect size f allows assessing an exogenous construct's contribution to an endogenous latent variable's R value. f values of 0.0, 0.15, and 0.35 indicate an exogenous construct's small, medium, or large effect, respectively, on an endogenous construct. Predictive relevance: Use blindfolding to obtain cross-validated redundancy measures for each endogenous construct. Make sure the number of observations used in the model estimation divided by the omission distance D is not an integer. Choose D values between 5 and 10. The resulting Q values larger than D indicate that the exogenous constructs have predictive relevance for the endogenous construct under consideration. The effect size q allows assessing an exogenous construct's contribution to an endogenous latent variable's Q value. As a relative measure of predictive relevance, q values of 0.0, 0.15, and 0.35, respectively, indicate that an exogenous construct has a small, medium, or large predictive relevance for a certain endogenous construct. For theory testing, consider using SRMR, RMS theta or the exact fit test. Apart from conceptual concerns, these measures' behaviors have not been researched in a PLS-SEM context in depth, and threshold values have not been derived yet. Following a conservative approach, an SRMR (RMS theta) value of less than 0.08 (0.1) indicates good fit. Do not use the GoF to determine model fit. Importance-Performance map. What constructs have a high impact on a target, and to what degree can it be improved?. References Hair, J. F., et al. (017). A primer on partial least squares structural equation modeling (PLS-Sem). Thousand Oaks, Calif, Sage.