discrepancy between Bayesian and "regular" repeated measures ANOVA

S_T

Hi,

We are new to Bayesian statistics and JASP, and got a result that looks weird to us - we assume that we have a problem with our interpretation.

Our design is a factorial design of 3X3X2 - all within subjects.

We performed both a repeated measures ANOVA and a Bayesian repeated measures ANOVA, and the results seem to differ.

In order to estimate the interaction, we divided the simplest model that includes the interaction with the model without it.
For example, the interaction between Ron and Hermione (which was significant p<0.001):
BF = Ron+Hermione+Ron*Hermione / Ron+Hermione =3.26e64 / 3.2e65 = 0.1,
which we interpret as evidence against the interaction.
(We attached the JASP results)

This seems strange to us. Did we estimate the effect correctly, or need to do something else?

Thank you very much!
S&T

EJ

Dear S_T,

Yes, the Bayesian results indicate evidence against the interaction. This is also evident from the posterior model probabilities [P(M|data)]. And I agree that this is unexpected, given the results from the classical analysis. These discrepancies can arise when N is really, really high (probably not the case here) or when the model assumptions have been violated.

Are the data normally distributed? Are the variances equal in the different conditions? You could construct a data set (by applying transformations etc) in which this is OK, and check whether the outcome is still different. If it isn't, you know that this is the source of the problem: model-misspecification.

As an aside, if you select "best model on top" and "BF_01", you might get a table that is easier to read.

Cheers,
E.J.