Three-way interaction Bayesian RM ANOVA

Niklas

Hi,

I don’t seem able to wrap my head around the model comparisons with three-way interactions in a RM ANOVA. I understand that for a two-way interaction, you divide the BF10 of the model with the main effects only by the BF10 of the model with both main effects plus the interaction term. However, I am not sure how to apply this to a three-way interaction. Specifically:

1. What model do I compare the three-way-interaction model to? Is it the model with all interactions to the second last model with all main effects and three two-way interactions? Setting everything as nuisance except the three-way interaction does the same job, correct?

2. In my case (see tables below), the best fitting model is one with two out of three possible three-way interactions. Do I always choose the best fitting model? Because if I compare it to a model with only one two-way interaction, the BF is ‘only’ 1.35 (379/280). Doesn’t this mean that the best fitting model is not really convincing compared to a less complex one? Where do I stop with the reduction process? In other words, how theoretically driven is the model selection process? And what balance do I strike between what the data tell me, parsimony, and theory?

3. Following this, what would I conclude about the three-way interaction I’m interested in? That it does not add much to the model (150/146); but that the model with all effects but no three-way interaction fares worse than one with only two two-way interactions (146/380). However, this best-fitting model with two two-way interactions itself does not provide strong evidence for the inclusion of those two two-way interactions compared to a simpler model with only one two-way interaction (379/280). Therefore, there is no strong evidence that the three-way interaction matters, and only weak evidence to deviate from the simplest interaction model?

Thanks a lot for a response!
Niklas

EJ

Hi Niklas,

Re. (1): Yes.
Re. (2): Good points. No the best model is not convincing. And I am not sure that it is correct to assess the three-way interaction by comparing the most complex model to the model that lacks the three-way interaction (because that model is not particularly good). An alternative approach is to average across the models and consider an analysis of effects.
Re. (3): Yes, correct. From the effects table you see that across all models, the BF is 2.356 for including the interaction.

Cheers,
E.J.