Welcome!

Sign in with your CogSci, Facebook, Google, or Twitter account.

Or register to create a new account.

We'll use your information only for signing in to this forum.

Supported by

Three-way interaction Bayesian RM ANOVA

Hi,

I don’t seem able to wrap my head around the model comparisons with three-way interactions in a RM ANOVA. I understand that for a two-way interaction, you divide the BF10 of the model with the main effects only by the BF10 of the model with both main effects plus the interaction term. However, I am not sure how to apply this to a three-way interaction. Specifically:

  1. What model do I compare the three-way-interaction model to? Is it the model with all interactions to the second last model with all main effects and three two-way interactions? Setting everything as nuisance except the three-way interaction does the same job, correct?

  2. In my case (see tables below), the best fitting model is one with two out of three possible three-way interactions. Do I always choose the best fitting model? Because if I compare it to a model with only one two-way interaction, the BF is ‘only’ 1.35 (379/280). Doesn’t this mean that the best fitting model is not really convincing compared to a less complex one? Where do I stop with the reduction process? In other words, how theoretically driven is the model selection process? And what balance do I strike between what the data tell me, parsimony, and theory?

  3. Following this, what would I conclude about the three-way interaction I’m interested in? That it does not add much to the model (150/146); but that the model with all effects but no three-way interaction fares worse than one with only two two-way interactions (146/380). However, this best-fitting model with two two-way interactions itself does not provide strong evidence for the inclusion of those two two-way interactions compared to a simpler model with only one two-way interaction (379/280). Therefore, there is no strong evidence that the three-way interaction matters, and only weak evidence to deviate from the simplest interaction model?

Thanks a lot for a response!
Niklas


RMAnova.JPG 119.5K
RMAnova2.JPG 29.6K

Comments

  • EJEJ Posts: 409

    Hi Niklas,

    Re. (1): Yes.
    Re. (2): Good points. No the best model is not convincing. And I am not sure that it is correct to assess the three-way interaction by comparing the most complex model to the model that lacks the three-way interaction (because that model is not particularly good). An alternative approach is to average across the models and consider an analysis of effects.
    Re. (3): Yes, correct. From the effects table you see that across all models, the BF is 2.356 for including the interaction.

    Cheers,
    E.J.

    Thanked by 1Niklas
Sign In or Register to comment.