Bayesian RM ANCOVA: no evidence for interaction but simple slope analyses indicates interaction
My model is a 2-level repeated within-subject factor (Repeated), a between-subject factor (Condition) and a covariate (Difference). Basically I want to look at the effects of Condition and Difference on the performances in 2 variants of a task (Repeat 1 and Repeated 2). I expect the effects to be different across the 2 variants.
When I ran a Bayesian RM ANCOVA in JASP all seems to make sense, except for that there was strong evidence against the interaction between Difference and Repeated.
The thing is when I plot the data I could see there's difference between the two lines. So I ran two separate Bayesian linear regressions to double-check. And indeed:
There's no effect of Difference on Repeated 1.
But there's an effect of Difference on Repeated 2.
My question is then: why did I find that there's no evidence for the interaction between Difference and Repeated in the RM ANCOVA?
Thank you for your help in advance!
Comments
Hi Spi,
This is my take on the issue: interactions are inherently complex --they can take on all kinds of shapes-- and this means that they spread their predictions out relatively widely; this entails a relatively strong penalty for complexity. Moreover, the evidence from the separate linear regressions is not overwhelming (Jeffreys would call is "not worth more than a bare mention" in both cases).
We are working to implement contrasts, and these ought to allow you to specify your expectations more precisely, so that there's less of a penalty for complexity. But it may take a while before we're perfectly happy with our solution.
Cheers,
E.J.