Difference between "Baws" Factor and BFinclusion

neelram

Dear JASP team,

Thank you for the great work on the program and adding the BFinclusion factor for RM ANOVAs. Because I was used to calculating a Bayes Factor for an interaction effect by hand (according to the "Bayes like a Baws" post), I did this first and then realized that the new BFinclusion features exists. Interestingly, I get different values for the same interaction effect with both methods. Therefore, I wanted to ask what the exact differences between both calculations are and which one might be more suitable or easier to interpret?

Thanks for the help!

EJ

The "Baws" method proposed by Sebastiaan should give the same result as "Analysis of effects" with the "matched models" option. Do you get a different outcome?
E.J.

neelram

Thank you for the quick help! It was my mistake because I had chosen the "across all models" option instead of "across matched models". Now, my calculations and BFinclusion yield the same result.

Regarding the interpretation of "across all models" and "across matched models" I am still a bit puzzled though.
For example, I want to look at the interaction of the between factor agegroup and within factor cue. I get a BFincl of 0.368 for the interaction effect "across all models" and 0.182 for "across matched models". So I would assume "across matched models" shows me that, in comparison to models without interaction, that this specific interaction effect does have substantial evidence for the H0. Hence, it is evidence for the cue effect in both agegroups being comparable and not significantly different. Or am I wrong here?
And what exactly would the "across all models" BFincl of 0.368 tell me for this interaction? Is it important for the question whether adding this interaction makes my whole model better or when would I need it?

EJ

Yes, you are correct.
The "across all models" BFincl is also not in favor of adding the interaction, although the strength of evidence is not as compelling as for the matched models. Perhaps the interaction term features in a few models with higher-order interactions that do relatively well but are excluded from the matched model analysis. It is also possible that the models with the single main effect do relatively well, and I think they are excluded from this matched model analysis as well (because you test the interaction).

E.J.

neelram

Alright, that makes a lot more sense now! Thank you!