Welcome!

Sign in with your CogSci, Facebook, Google, or Twitter account.

Or register to create a new account.

We'll use your information only for signing in to this forum.

Supported by

RM ANOVA - extrem diverging main effects - interpretation of the interaction effect

Hello!
I have a question regarding a conducted repeated measures ANOVA.
My first factor 1 has extrem evidence for H1 (BF10 > 7.000e+19) and my second factor 2 has moderate evidence for H0 (BF10 = 0.147). BFinclusion for the interaction factor1*factor2 = 0.145.
Is the low interaction BF in this case probably misleading because of the high factor1 and the low factor2?
How should I interpret the interaction effect? Is it right when I just say that the interaction effect shows moderate evidence for H0, i.e. that no interaction exists?

Thank you very much!
Best,
Markus

Comments

  • EJEJ Posts: 409

    Hi Markus,

    In JASP we use the marginality principle, which means that when an interaction is present, so are the constituent main effects. The most straightforward test is to compare the full model against the model with only the main effects. When one or both main effects do not receive a lot of support, you can wonder whether this is the right comparison, and the inclusion probability may make more sense. In your case, both will probably point to H0, so yeah, I think you have some evidence in favor of H0.

    Cheers,
    E.J.

    Thanked by 1Markus
  • MarkusMarkus Posts: 15

    Great, thank you very much E.J.!
    Best,
    Markus

  • MarkusMarkus Posts: 15

    Hi,
    I have a problem that I can not handle. I found a significant interaction effect (p < .02) based on classical repeated measures ANOVA analysis but I got a insensitive BFinclusion = 1.6 result based on Bayes analysis. Now my biggest problem is
    how I should handle this in a paper. When I would publish the results without the BFs everything would be fine, but no I have to find the right words when I include the BFs. Because now a reviewer could possible say that my study was not designed well enough to find the effect and rejects the paper.

    Another problem here is that I can not explain what the prior in the repeated measures ANOVA really means (r scale fixed effect of: 0.5; r scale random effects: 1; and r scale covariates: 0.354).

    Thanks for any advice or help!
    Best,
    Markus

  • EJEJ Posts: 409

    I was a bit caught up and did not respond too quickly. I have now responded to a similar question you just asked. Sorry about my tardiness. I try to answer quickly but sometimes diapers and deadlines get in the way.
    Cheers,
    E.J.

  • FSellaFSella Posts: 5

    Hi,

    I saw the discussion and I thought to join in.

    I am new to Bayesian analysis and I would like to have a feedback on the Bayesian ANOVA I have just run (see attached image). I am interested in assessing the evidence for the model that includes the interaction term. The model with the interaction Time * Group has a BF10 = 2.120e+7 whereas the model with the two main effects Time + Group has a BF10 = 639811.912. It is my understanding that the comparison of the model with the interaction and the model with the two main effects can be achieved with a simple division (due to transitivity): BF10 Interaction / BF10 main effects (i.e., 2.120e+7/639811.912 = 33). The BF = 33 suggests a strong evidence in favour of the model with interaction. Am I right?

    Thanks in advance for any help.

    Francesco

    ANOVA.PNG 11.3K
  • EJEJ Posts: 409

    Hi Francesco,
    Completely correct!
    E.J.

  • FSellaFSella Posts: 5

    Hi EJ,

    thanks for the immediate reply.

    We have analysed the performance of two small clinical groups (6 training vs 6 controls) across nine time points. Again, it is my understanding that Bayesian approach is more appropriate with small sample sizes as long as the priors (we used the default in JASP) are reasonable (with large large samples likelihood tends to dominate whereas priors may have a stronger influence with small sample sizes). Now, I am planning to run a series of Bayesian t-tests (still in JASP) in order to obtain a BF for each comparison (training vs. control for each time point). I would also double check the results using the BEST package to obtain a posterior distribution of the difference between groups' means.

    However, the Editor that handles this paper asked us to run a classic mixed ANOVA (Ws:Time; Bs: Group) with bootstrapping along with non-parametric analysis (Mann-Whitney) for the post-hoc comparisons. Is there any motivation to use Bayesian analysis over NHST in case of small sample sizes, or would it be better in this case to not use Bayesian analysis?

    Thanks in advance

    Best wishes

    Francesco

  • EJEJ Posts: 409

    Hi Francesco,

    It is always a good idea to use Bayesian inference. But at the very least you can argue that the two paradigms are complementary, and it is counterproductive to ignore the Bayesian outcomes. So my advice is to heed the editor's advice, but report both.

    Cheers,
    E.J.

  • peterose1peterose1 Posts: 2

    Is the result of Francesco's division (BF10 = 33) strong evidence of a pure interaction effect? It seems to me to state only that a model including the interaction term is more strongly supported than one without it.

  • EJEJ Posts: 409

    Hi peterose1,

    JASP does not allow "pure" interaction effects (meaning without the constituent main effects). This is the principle of marginality, discussed in part II available at https://osf.io/ahhdr/

    Cheers,
    E.J.

  • peterose1peterose1 Posts: 2

    Thank you. I am downloading he paper now. But meanwhile, I conclude my other interpretation is correct: "a model including the interaction term is more strongly supported than one without it." Yes?
    (I am BoomChicago dad.)

  • EJEJ Posts: 409

    Yes, that's correct.

Sign In or Register to comment.