Bayes factor for the two-way interaction: model comparison vs. raw effect test
Hi everyone!
I’m interested in the two-way interaction effect. Using BayesFactor package, I compared main effects model with interaction vs. main effects only model (divided one BF by another). I used default priors. And I got substantial evidence for the main effects only model (BF10 = 0.13).
However, a reviewer asked me to use informed priors - as we potentially have an estimate of the expected interaction. I estimated raw interaction size from the previous study using 'emmeans' package. Then I calculated the size of interaction (and it’s SE) in my current study. Then I applied Z. Dienes’s BF calculator here: http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inference/Bayes.htm (half normal with SD = raw interaction as a prior for H1; current interaction size and it’s SE as empirical data). The calculator provided me with BF10 = 0.46. That is, saying that I don’t have enough data in favour of the null.
I understand that in both cases I have some evidence against the interaction, and these numbers just appeared to be on the different sides of the arbitrary boundary. However, it seems that these are two very different approaches. Which one seems more reasonable to you? What should I use in this situation?
Thanks in advance!
Comments
I'd report both and acknowledge the uncertainty. Generally it is a good idea to include more knowledge. In particular, if the observed interaction is qualitatively consistent with the predicted interaction, this should help the model with the interaction.
E.J.
Thanks for the response! Does it mean that both approaches are equally valid?
It is a matter of taste, imo
Ok, thank you!