Bayes Correction for multiple testing

obemar · February 2021

Hello there,

As Post Tests for a bayesian rm-ANOVA I conducted a bayesian dependent t-tests with a BF correction as proposed in this feed and the papers mentioned there (de Jong, and van den Bergh et al.)

Since I had 4 comparisons I multiplied the BFs by 0.4142.

And here lies my problem. For one comparison I receive a BFcorr of 0.07 (BF10uncorrected = 0.173), indicating a positive (Raftery) or strong (Jeffreys) support for H1. But the means of this comparison are literally the same (M = 497) and the SDs pretty much too (48 vs 46).

So my question is whether or not with this correction only the "positive side" of BFs count. Or do I not correct BFs between 1-3 and 1-0.33?

This is the first time I work with Bayes Factors and JASP and the error could possibly lie within my settings, since I do not understand everything yet. I just opened a bayesian paired t-test, entered my variable pairs and left all settings at the default (Measure 1 =/= Measure 2, BF10, Cauchy scale = 0.707 ).

I do not get any prior or posterior odds in the output, which could help me with the interpretation.

If need be, I can provide the dataset or more detailed information.

Thanks for any help or advices,

Maria

EJ · February 2021

Hi Maria,

Your BF10uncorrected = 0.173, meaning that the data are 1/0.173 = 5.78 times more likely under H0 than under H1. In other words, BF01uncorrected = 5.78, meaning you have evidence in favor of H0. The multiplicity correction increases that evidence to 1/.07 = 14.29.

JASP generally outputs the BF. You can specify your own prior odds, and then multiply these by the BF to obtain your own posterior odds.

Cheers,

E.J.

obemar · February 2021

Hi,

Thank you for your answer, @EJ . I'm trying to wrap my head around this.

So basically rather than looking at those BF-size tabels like Jeffrey's I just have a look at the likelyhood of H1 to H0 (or vice versa). And the higher this likelyhood under BF10 is, the more evidence I have for H1. The higher the likelyhood under BF01 is, the more it speaks for H0. And if under BF10 the likelyhood is small, it also speaks for H0 (and for BF01 for H1)?

So of my other corrected BFs two are in favour of H1 (BF10corrected = 1672.246, and 4.214) and one in favour of H0 (BF10corrected = 0.374)

Did I get it right?

Thank you for your help,

Maria

EJ · February 2021

A BF is like the scales of Lady Justice -- when one scale goes up the other one must go down; they are in a one-to-one correspondence. So if you have BF10 = 5 the data are 5 times more likely under H1 than under H0, and this is evidence for H1. This statement is identical to BF01 = 1/5. The BF equation is

BF10 = p(data | H1) / p(data | H0) so

BF01 = p(data | H0) / p(data | H1);

We are just switching numerator and denominator.

Cheers,

E.J.

Howdy, Stranger!

Categories

Bayes Correction for multiple testing

Comments

Howdy, Stranger!

Quick Links

Categories

Bayes Correction for multiple testing

Comments