Baysian Model Averaged Meta Analysis Using JASP - Most appropriate inputs

PeterAllen

Hello.

I've run a series of pre-registered replications, analysed using Bayesian independent samples t-tests. I now want to do a Bayesian mini-meta, following the tutorial by Berkhout et al (2023): https://doi.org/10.3758/s13428-023-02093-6 

Ideally, I want to report the results of the individual comparisons and the meta-analysis in a single table that looks something like this (it's just the forest plot without the visualisation of the effect sizes):

Anyway, I initially ran the meta-analysis using the effect sizes (delta) and credible intervals provided with the individual Bayesian t-tests in JASP. These effect sizes and BCIs, of course, get reproduced in the meta-analysis forest plot, along with the meta analytic effect sizes and associated credible intervals.

However, when Berkhout and colleagues describe the process (top of second column on p. 6), they use Cohen's d (i.e., the sample stat rather than population estimate) and its 95% confidence intervals (or SEs). If I follow this, these Cohen's d figures and the frequentist confidence intervals end up on the forest plot, along with the meta analytic effect sizes and associated credible (I assume) intervals.

Now, in a practical sense, it doesn't really make much difference. Both sets of inputs lead to numbers that are similar enough for there to be no differences in terms of interpretation. However, I'm assuming that one approach makes 'more sense' than the other. And I'm also keen to get the individual comparisons and the meta-analytic results in the same table, which (I think) I can only really do if I use delta/BCIs as the inputs.

So, the question is: I want to do a Bayesian model-averaged meta-analysis of several replications which all boil down to pairwise comparisons. As the inputs/data for the meta-analysis, should I use the the effect size estimates (delta) and Bayesian credible intervals from JASP's Bayesian t-test output (these were pre-registered as the analysis for the individual studies/replications), or should I do a parallel set of frequentist t-tests, calculate Cohen's d and associated confidence intervals for each of these, and then use these as the inputs/data?

Thanks in advance! Best wishes, Peter Allen

PeterAllen

Hello! I'm going to bump this in the hopes that someone is able to provide some advice :)

EJ

Sorry about the tardy response. I've asked our expert. In my opinion, it makes more sense to enter the frequentist results, because otherwise the prior is used multiple times (but I have not made up my mind about this)

E.J.

PeterAllen

Thanks EJ. I shall run with this!

František

Hi Peter,

Yes, EJ is indeed right. If you input the Bayesian posterior estimates, you are adding additional information to each study via each t-test's prior distribution.

The `frequentist t-test's` Cohen's d is just a summary statistic used as an input to the meta-analysis (instead of the raw individual observation) -- so there is nothing frequentist/Bayesian about it :)

Cheers,

Frantisek

PeterAllen

Thanks Frantisek. Yes, of course ... this all makes sense now! Regards, Peter Allen