Bayes vs Baws
Suppose one has a 3-way mixed design with 2 repeated measures, a and b, and one group measure, c, and one is interested in the BF10 for the ab interaction.
If it were just a simple 2-way repeated design, (with no c) then the BF10 for the ab interaction would be the BF10 for (a+b+ab) divided by that for just (a+b). If the former = 20 and the latter = 10, then the quotient = 2. This means that the term including the interaction effect is twice as likely as the one without it. But the one without it is 10 times as likely as the null so since 210 = 20, the BF10 for (a+b+ab) is 20 times as likely as the null. Does this mean that since the BF10 for the interaction by itself is 2, it follows that a putative model with just the interaction is twice as likely as the null? Many people proceed in this way…
However, we have read that the principle of marginality requires that one should not speak of a “pure” interaction effect in the absence of main effects. This always strikes me as puzzling since a perfect 90 degree crossover interaction would yield no main effects but could very well yield an interaction in a standard ANOVA. ( 1. Comments?)
Assuming for now that the interaction BF10 (noted above) by itself is 2,meaning the model with the interaction is twice as likely as the null, what happens when one expands the model to include the between groups c effect? SEBASTIAN has described “Baws factor” procedures to deal with this issue, which entail dividing the sum of all the BF10’s for terms with the interaction by the sum of the BF10’s for terms lacking it. He notes that the result is often similar to the BF10 for the interaction based just on a and b. I have observed the same thing: The Baws and the simple Bayes rarely differ by an order of magnitude. That is, they rarely differ by an amount that would change their interpretation from say “anecdotal” to “substantial” (Jeffries, 1961).
It therefore seems to me that one should stick with the simple BF10 for (a+b+ab) divided by that for (a+b). After all, the interpretation of the quotient remains as described above, with or without other added independent variables, no?
(2. Comments?)
Comments
Dear peterosel,
Yes, we don't deal with models that include only the interaction term. One of the reasons is that it is a priori rather unlikely to have a perfect crossover interaction. I believe that there is another argument based on transformations, such that perfect crossover interactions are immediately destroyed by any nonlinear transformation. There are references in the "Part II" PBR paper (https://link.springer.com/article/10.3758/s13423-017-1323-7).
As far as the test of the interaction is concerned, I am not sure that there is a single correct was of doing this. There are multiple models in play and the decision which ones are the relevant ones to compare is not always 100% clear. So on general grounds I would argue that it is nice to show robustness and examine the same question from different angles.
Cheers,
E.J.