Bayesian Repeated Measures ANOVA

ABC_95

Hi,

In a frequentist repeated measure ANOVA I found a significant main effect of difficulty (p < .001) and no significant main effect of drink type (p = .798) or interaction between difficulty and drink type (p = .973). I'd like to further this by conducting Bayesian analyses, can someone double check my interpretation of the output below?

Extremely (?) strong evidence in favour of a main effect of difficulty (p <.001; BF10 = 723439.24). Anecdotal evidence in favour of the non-significant main effect of drink type (p = .798; BF10 = 0.15). Inconclusive evidence to support the non-significant interaction effect between difficulty and drink type (p = .973; BF10 = 0.09).

The BF10 for the interaction was calculated by: 9518.224 / 108820.390 = 0.08746728 (and then I rounded this to two decimal places).

Thanks very much for any advice!

To finish: Out of curiosity, what difference does it make to under 'order' select 'compare to null model' versus 'compare to best model' - is there a gold standard option to select here? For now I have compared to null model.

EJ

Hi ABC_95,

If you select "compare to best model" and tick "BF01" then you'll immediately see how much better the best model predicts compare to its competitors. The comparison to the null is often not so interesting, because it performs so poorly. You can also add difficulty and drink type to the null model, and you'll immediately see the BF for adding the interaction in the table (instead of computing it).

With respect to your interpretation:

1. "Anecdotal evidence in favour of the non-significant main effect of drink type (p = .798; BF10 = 0.15)" The data support the absence of the main effect of drink type; the data are 1/0.15 = 6.67 times more likely under the null model than under the model with only drink type. Note that numbers below 1 are not weak evidence; they are evidence in favor of the other hypothesis. So 1000 for H1 is the same as 1/1000 for H0. I recommend that when BFs are lower than 1, you invert them before interpreting.
2. "Inconclusive evidence to support the non-significant interaction effect between difficulty and drink type (p = .973; BF10 = 0.09)." Same issue as above, you want to invert this. So you have BF = 11.43 in favor of the absence of adding the interaction on top of the two main effects.

Cheers,

E.J.

ABC_95

Hi E.J.,

Thanks for the response, that was super useful.

When I click 'compare to best model' the BF for 'difficulty' changes from 723439.241 to 1.00, and a new row called 'null model' has the large BF of 721567.989.

I know that the main effect of 'difficulty' is significant (p<.001), but regardless of whether I select BF10 or BF01 the BF for 'difficulty' when 'compare to best model' is always 1.00 and I am finding this difficult to interpret. How does this support the significant p<.001 finding? In fact, with any data I use, when I tick 'compare to best model' the first row is always 1.00.

Does this mean that the previous BF for 'difficulty' of 721567.989 when I selected 'compare to null model' was inaccurate? I did think the number was very large given that BF of over 100 is considered extreme evidence. I am confused about how to establish to BF for difficulty.

Any advice very much appreciated,

Thanks

EJ

Hi ABC_95,

The top entry is always 1, since the model on the first row is compared to itself. So the model with only "difficulty" is best, predicting the data 6.67 times better than the model that adds image type, and predicting the data about 77 times better than the model that adds both image type and the interaction between difficulty and image type, etc.

E.J.