How to find the BF of a main effect when the best model includes interactions.
Hello. I am using anovaBF from the BayesFactor package for R. Usually to find the BF of a main effect, I compare the best model with this main effect, against the model that has the same main effects and interactions except for the one I am looking for.
I want to find the BF for the main effect of A, but the best model [18] contains also two interactions (A:C and A:B) and the best model without A (4) has no interactions. If I comparte 18 against 4 (sort(Anova)[18]/sort(Anova)[4]) I get a BF of 1.587706e+14, but it includes the interactions.
The other option would be 12 (with only main effects) against 4, which gives 1.606811e+12.
> sort(Anova) Bayes factor analysis [1] B + S : 35319663 ±0.41% [2] C + S : 35932312 ±0.2% [3] C + B + C:B + S : 1.107802e+13 ±0.81% [4] C + B + S : 9.975825e+13 ±0.58% [5] A + S : 1.510931e+15 ±0.31% [6] A + C + S : 2.025998e+18 ±1.27% [7] A + C + A:C + S : 1.408867e+19 ±0.76% [8] A + C + B + A:B + C:B + S : 1.098129e+25 ±4.21% [9] A + C + B + C:B + S : 1.935634e+25 ±0.93% [10] A + C + B + A:B + S : 7.503825e+25 ±3.19% [11] A + B + A:B + S : 9.598767e+25 ±1.01% [12] A + C + B + S : 1.602926e+26 ±1.86% [13] A + C + A:C + B + C:B + S : 2.149663e+26 ±1.14% [14] A + B + S : 2.969097e+26 ±1.38% [15] A + C + A:C + B + A:B + C:B + A:C:B + S : 6.400354e+26 ±1.47% [16] A + C + A:C + B + S : 1.63816e+27 ±1.27% [17] A + C + A:C + B + A:B + C:B + S : 2.496953e+27 ±1.14% [18] A + C + A:C + B + A:B + S : 1.583868e+28 ±0.84%
Then to find the BF of A:B I would compare 18/16, and for A:C it would be 18/10?
Which option is better and why? Do I need to use my best model to find the BF of each main effect and interaction, or can I compare two models that allow me to make a fair comparison even if I don´t use the best model?
Thanks.
Comments
Hi Aram,
I think this is one of the few issues where Richard and I have a different opinion. I would argue that there are multiple models to consider, and it is best to average over them. In JASP, you can do this by ticking "Effects", and then the version suggested by Sebastiaan, "across matched model". Sebastiaan initially called this the "Bawes" factor -- if you look for that in this forum you should find the relevant posts.
Basically, what you get in this analysis is a comparison of all model with the effect to all models without the effect (but matched such that interactions are excluded when they are not common to both models)
Cheers,
E.J.