Conflicting results from RM ANCOVA and Bayesian RM ANCOVA

FSella

Hi,

I would really appreciate any help with the following issue.

I have contrasting results from RM ANCOVA and Bayesian RM ANCOVA and I would like to have a feedback on my reading of the outputs.

There is a significant interaction Stim x Hand using the RM ANCOVA (there seems to be a three-way interaction as well but for the time being let's focus on the two-way interaction).

Then, I run a Bayesian RM ANCOVA on the same data and here it is the output:

Given that I am interested in the interaction Stim x Hand, my next move is comparing the model with the main effects (Stim + Hand + Order) vs. the model with the same effects and the interaction (Stim + Hand + Stim*Hand + Order) --> BF10 main effects and interaction / BF10 main effects --> 0.1/0.015 = 6.67.

I would say that the model including the interaction term is more in favour of H1 (moderate evidence) compared to the model without interaction. However, the BF10s are all less than 1!
Therefore, the model with interaction does not support H1.
Here it emerges the discrepancy with the RM ANCOVA.

I have the feeling that I am missing something here.
Could anyone help?

Many thanks in advance

Best wishes

Francesco

EJ

Hi Francesco,

Great example. You summarize the results well. So if you already have the main effects in there, there is some evidence for including the interaction. But at the same time, the null model beats all of the other models. As always, I recommend complete transparency. How serious you take the evidence for the interaction depends on how you feel about including the main effects, for instance "as nuisance" -- say you already knew your main interest is for the interaction, and you know that the principle of marginality demands that you include the constituent main effects for any interaction, then you might argue that your point of departure is from the main effects model (JASP allows you to include these "as nuisance" directly). If you approached the data set more with an open mind, then I'd consider all models and the evidence is not compelling for either.

Cheers,
E.J.