Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Prior in Bayesian repeated measures ANOVA

I have a two by two by two factorial design in which I tested whether there was an interaction. In the 'standard' rmANOVA I observed no evidence for a reliable interaction and this was confirmed by a Bayesian Analysis demonstrating that the two main effects model was the best model. Adding the interaction resulted in a BF10 of 0.41. I thus concluded that there was only weak evidence against a reliable interaction (2.44). This inspired a replication where I again observed no interaction and a BF of 0.281. This experiment thus provided stronger evidence against an interaction but still not overly convincing (3.56). I was wondering whether, and if so how, it is possible to include exp 1 as a prior in the analysis of Exp 2???

Thanks for the help.

Cheers,

Dirk van Moorselaar

Comments

  • Hi Dirk,

    There are two ways to do this. First, there is the Verhagen & Wagenmakers method, where you "simply" use the posterior from the first experiment as a prior for the second experiment. Unfortunately, the updating and specification process are non-trivial for ANOVA models. Second, you can use the Ly method and obtain a BF by adding the data together. This will yield the same result as updating one batch at a time. The Ly method does assume that the data are exchangeable, so the replication is as exact as can be. If you Google "replication Bayes factor" on my website you should find the relevant papers.

    Cheers,
    E.J.

  • Hi Erik Jan,

    Thank you for the very fast reply. This means I might be able to resubmit before Christmas :). I read your paper and this is where I got a bit confused. These are the Bayes factors that I obtained. Note that the factor always denotes how much evidence there is against the interaction being a better model than the two main effects model:

    Exp 1: BF10 = 0.41 -> 2.44
    Exp 2: BF10 = 0.281 -> 3.56
    EXP 1+2: BF10 = 0.194 -> 5.15

    If I understand your paper correct I can obtain the bayes factor of experiment 2 where experiment1 is included as a prior as follows: 5.15/2.44 = 2.11.
    So if I did everything correct, this means that the bayes factor now becomes smaller despite including exp 1 as a prior than it is without a prior. Maybe I am misunderstanding something but this feels a bit counterintuitive to me, because exp 1, although not very reliable, already demonstrated some evidence against the interaction

    Thanks again,

    Dirk

  • Hi Dirk,

    Exp 1 provided some evidence against the interaction; consequently, in the model that includes the interaction, the corresponding posterior distribution will have more mass near zero than the prior did. In other words, the interaction --if it exists-- is now known to be relatively small. Effectively, after seeing the data from Exp 1, the interaction model now makes predictions that are relatively similar to the two-main-effects model. When models start to make similar predictions, the evidence decreases.

    Cheers,
    E.J.

  • Hi Erik,

    Thanks this is very clear. Enjoy the Christmas break!

    Cheers,

    Dirk

Sign In or Register to comment.