Bayes Factor comparison

Jaiden_Camryn

Hello,

In two studies, I have replicated (experiment 1) and expanded (experiment 2) by adding a condition for comparison the work of another researcher.

My aim is to test whether the replicated condition in experiment 1 conserves the same effect size or is reduced by the introduction of the new variable in experiment 2.

In both experiments, I have computed the BF using both the Cauchy default and Oosterwijk prior distribution for a Bayesian one-sided paired-samples t-test.

In experiment 1, I obtained a Cauchy BF10 = 5.77 (median=0.51; 95CI[0.13-0.98]) and a Oosterwijk BF10 = 10.93 (median = 0.39; 95CI[0.19-0.67].

In experiment 2 when isolating the replicated condition from the expanded one I obtained a Cauchy BF10=3.00 (median=0.24; 95CI[0.05-0.45] and Oosterwijk BF10=7.67 (median=0.30; 95CI[0.13-0.45]).

What I would like to know is if there is a quantitative way to test this reduction in the value of the Bayes Factor for the variable of interest between the replicated study and the expanded one.

Thank you very much in advance,

Cheers,

JC

EJ

Hi JC,

Sorry about the tardy response, this one slipped through. First of all, there is no way to compare BFs directly. Instead, you need a BF for the difference. So what you'd need to do is combine the two conditions into one analysis file, and test for an interaction.

Cheers,

E.J.

Jaiden_Camryn

Hello EJ,

Thank you very much for your response. Can I still combine the two conditions into one analysis file, and test for an interaction if the sample sizes between the two studies are very different (n1 = 20, n2 = 90)?

Thanks in advance,

Cheers,

J.C.

EJ

Yes, the Bayesian ANOVA can also be applied to unbalanced designs, so this should not be a problem.

Cheers,

EJ

Jaiden_Camryn

Splendid!

I find the Bayesian logic a bit tricky to apply however. Should I look at the BFinclusion or BFexclusion?

When I use a Bayesian RM anova for BF10, I get a BFincl = 0.384, which means, if I understand correctly, that the BF for the difference between replicated and expanded conditions is very small and there is no support for the alternative hypothesis (difference between conditions), but I am not sure there is sufficient support for the null hypothesis either (BFexcl = 1/0.4 = 2.5). Does this mean that none of the hypotheses is supported?

Thanks,

Cheers,

J.C.

EJ

It means that the data are not sufficiently diagnostic for the purpose of discriminating the models -- all models retain a non-negligible amount of posterior probability.

E.J.

Jaiden_Camryn

Hi EJ,

Thanks a lot for all your explanations.

Cheers,

JC