Modification indices and Factor loadings: CFA beginner asking for help!

gvleioras

Dear JASP team,

I have translated a 27-item questionnaire in Greek and have administered it to 200 students. Now, I wish to check if the original factorial structure replicates in my data. Therefore, I choose a CFA. The questions:

1. Are the loadings in the Parameter estimates: Factor loadings table standardized or unstandardized? If they are unstandardized, how can I calculate the standardized ones? If they are standardized, how can they be larger than 1 (this occurs if I conduct the analysis on parcels).
2. In a publication with the same aim as my study, the authors describe modifying their original model as follows: "We deleted items where the modification indices indicated standardized loadings (i.e., 5% of variance) on a non-target factor". I assume this could be seen somehow in the Modification indices: Cross-loadings table, right? But how do Mod. Ind. and EPC show me what I am looking for?
3. In the same publication as in (2), the authors also modified the original model as follows: "We allowed covariances between items within the same factor where modification indices suggested standardized values .224 (i.e., 5% of variance)". I assume this could be seen somehow in the Modification indices: Residual covariances table, right? But how do Mod. Ind. and EPC show me what I am looking for?
4. Finally, what does a negative EPC signify? That the fit indices will decrease if I do something with the respective issue in my model?

Many thanks!

Georgios.

JMBKoch

Hey Georgios,

1. The loadings are unstandardized. You can get the standardized ones by choosing a standardization (usually you would go for "All") in the Advanced option.
2. So let's first disentangle what modification indices are. Modification indices indicate the change in the Chi-square of model fit if we were to include the parameter in question (e.g. a cross-loading, a residual covariance) into the model. Can you see how the default threshold to display the modification indices is 3.84? Thats' because a Chi-square difference test (with 1 df) is significant when the Chi-square difference is 3.84. Hence, all modification indices that are shown in the table represent modifications that would significantly improve the model fit. Now, the EPC is the expected parameter change. So this doesn't refer to the global model fit but to the estimate of the parameter itself. For instance, if the EPC of a cross-loading that was previously be fixed to 0 is 0.772, the cross-loading will be estimated as 0.772 (= 0.772 larger than 0) when we free it in a new model. Now, with that out of the way, I'm assuming that the authors of the paper you refer to looked at the standardized EPC (the EPC you get if you fit the standardized model), and in a subsequent step freed the cross-loadings whose squared EPC was larger than .05 (the square of the standardized loadings can be interpreted as the proportion explained variance of the item by the latent variables). The sentence as you cite it is odd, since a standardized factor loading itself cannot be interpreted as a proportion explained variance. I'm only trying to make sense of it, but I don't quite know what the authors actually did here. Now, most importantly, I actually do not believe that you should engage in such a step at all. IMHO, the modification indices should only be used to adjust the model, if the global fit of the model is bad, and it isn't an option to improve the model based on theory and previous research. The reason for this, is because the modification indices are based on your data (including all of the sampling error and bias associated with your data). Hence, if you rely on them too much, you may end up adjusting your model in a way that only works for your data, but not for your research population in general. This is called 'overfitting' and should be avoided at all costs. One way of attempting to avoid overfitting is not relying on modification indices unless strictly necessary. Now, of course, that doesn't mean that modification indices are entirely useless, but you should not let them guide you blindly. For instance, if a modification index indicates that there is a cross-loading, and you think that theoretically, this cross-loading makes sense it's a different story than finding that the EPC crosses a certain (in practice probably quite arbitrary) threshold.
3. See my answer in 2. I can't quite make sense of this sentence either for the same reason as above, but it's hard to say more without having seen the whole paper. IMO the only proportion explained variance would be the square of the standardized factor loading. I have no idea what they could mean with "5% of the variance" in the context of the residual covariances.
4. A negative EPC means in your case that the parameter would be negative if we add it to the model (e.g. a EPC of a cross-loading of -.02 would mean that the cross loading is -.02 smaller than 0, hence -.02. This is of course assuming that your cross-loadings are initially set to 0).

Best,

Michael