Accounting for multiple comparisons - Bayesian t-tests

kcourey

Hello,

I understand how JASP adjusts ANOVA post-hoc test priors for family-wise error. However, is there a way to correct for multiple comparisons for Bayesian t-tests? If JASP does not provide this functionality, is there justification for not correcting for multiple comparisons within a Bayesian framework or is there a way to do it manually?

Best,

KC

EJ

The method for correction can be the same as that applied for post-hoc tests in ANOVA. See the thesis by Tim de Jong for details: https://psyarxiv.com/s56mk

Cheers,

E.J.

dilcetan

Hi,

Is there a way to adjust the credible intervals for the comparisons too? Or is it okay to report unadjusted credible intervals?

Regards,

DT

EJ

Hi DT,

Ah, this is an interesting question (and sorry about the tardy response). I would say that the credible intervals that are reported are conditional on H1, and therefore do not need a correction. However, the unconditional credible intervals involve model averaging over H0 -- see https://journals.sagepub.com/doi/full/10.1177/2515245921992035. With more comparisons in play, the prior weight on H0 increases, and this draws the model-averaged estimate toward zero. Actually, I am not 100% certain how to compute the 95% credible intervals for a mixture between a spike at zero and a continuous slab -- it may just be better to report these in full.

Cheers,

E.J.