Is a t-based BIC Bayes factor approximation reasonable for three-level meta-regression?
Hello,
I am conducting a meta-analysis that aims to compare mean values of a continuous variable X between two study types, A and B. The variable ranges roughly from 10 to 100. I also have a few study-level moderators, such as mean age and proportion of females.
I ran a meta-regression using metafor. The model is three-level, with samples nested in studies (although only a few studies include more than one sample). It is a random-effects model, fitted with REML and based on t-statistics:
res <- rma.mv( yi = yi, V = vi, mods = ~ Group + other_moderators, random = ~ 1 | Study_ID/Sample_ID, method = "REML", test = "t", dfs = "contain", data = dat )
Group is coded as a factor comparing study type A vs B.
Since I received mostly null findings, I would like to add a simple Bayesian framework to help interpret them. I am thinking about using a BIC-based Bayes factor approximation:
BF_10 ≈ exp((t^2 - log(k)) / 2)
where (t) would be the t-statistic for the group coefficient from my model, and (k) would be the number of samples.
Do you think this is reasonable in this context? Are there any important pitfalls?
Thank you
Comments
Hi @SzTy
I'm afraid I'm not familiar with as it were "translating" the results of a classical meta-analysis into its Bayesian analogue, in the way that you suggest, but there again I am only another JASP user! :-) Along with clients, it appears that the development team have directly implemented Bayesian meta-analysis including meta-regression in JASP. This will certainly directly get you your required BFs, while following what, I'm sure is the gold standard methodology.
Bartoš, F., & Wagenmakers, E. J. (2025). Meta-Analysis with JASP, Part II: Bayesian Approaches. arXiv preprint arXiv:2509.09850. https://arxiv.org/abs/2509.09850
I do hope this helps, and I'm sorry I couldn't answer your question more directly! But perhaps somebody else will come along and do so!
Best,
Tarandeep