Modification indices and Factor loadings: CFA beginner asking for help!

Comments

• JMBKoch

  August 2021

  Hey Georgios,

  
  

  1. The loadings are unstandardized. You can get the standardized ones by choosing a standardization (usually you would go for "All") in the Advanced option.
  2. So let's first disentangle what modification indices are. Modification indices indicate the change in the Chi-square of model fit if we were to include the parameter in question (e.g. a cross-loading, a residual covariance) into the model. Can you see how the default threshold to display the modification indices is 3.84? Thats' because a Chi-square difference test (with 1 df) is significant when the Chi-square difference is 3.84. Hence, all modification indices that are shown in the table represent modifications that would significantly improve the model fit. Now, the EPC is the expected parameter change. So this doesn't refer to the global model fit but to the estimate of the parameter itself. For instance, if the EPC of a cross-loading that was previously be fixed to 0 is 0.772, the cross-loading will be estimated as 0.772 (= 0.772 larger than 0) when we free it in a new model. Now, with that out of the way, I'm assuming that the authors of the paper you refer to looked at the standardized EPC (the EPC you get if you fit the standardized model), and in a subsequent step freed the cross-loadings whose squared EPC was larger than .05 (the square of the standardized loadings can be interpreted as the proportion explained variance of the item by the latent variables). The sentence as you cite it is odd, since a standardized factor loading itself cannot be interpreted as a proportion explained variance. I'm only trying to make sense of it, but I don't quite know what the authors actually did here. Now, most importantly, I actually do not believe that you should engage in such a step at all. IMHO, the modification indices should only be used to adjust the model, if the global fit of the model is bad, and it isn't an option to improve the model based on theory and previous research. The reason for this, is because the modification indices are based on your data (including all of the sampling error and bias associated with your data). Hence, if you rely on them too much, you may end up adjusting your model in a way that only works for your data, but not for your research population in general. This is called 'overfitting' and should be avoided at all costs. One way of attempting to avoid overfitting is not relying on modification indices unless strictly necessary. Now, of course, that doesn't mean that modification indices are entirely useless, but you should not let them guide you blindly. For instance, if a modification index indicates that there is a cross-loading, and you think that theoretically, this cross-loading makes sense it's a different story than finding that the EPC crosses a certain (in practice probably quite arbitrary) threshold.
  3. See my answer in 2. I can't quite make sense of this sentence either for the same reason as above, but it's hard to say more without having seen the whole paper. IMO the only proportion explained variance would be the square of the standardized factor loading. I have no idea what they could mean with "5% of the variance" in the context of the residual covariances.
  4. A negative EPC means in your case that the parameter would be negative if we add it to the model (e.g. a EPC of a cross-loading of -.02 would mean that the cross loading is -.02 smaller than 0, hence -.02. This is of course assuming that your cross-loadings are initially set to 0).

  
  

  Best,

  Michael