Computation of posterior probability in favor of H0 [p(Ho/D)]

JH89

Hi everyone,

if I understood this correctly, the Bayes Factor, BF [p(D/Ho) : p(D/H1)] quantifies the relative evidence in favor of the null hypothesis over the alternative hypothesis. The BF can be directly computed in JASP.

However, I was wondering whether JASP does also directly compute the probability of the Ho given the obtained data [p(Ho/D)]? If not, is there a way to compute this probability given the BF?

Thank you
Jonas

EJ

Hi Jonas,
With the BF in hand (say 3 in favor of the null), you need to determine your own prior model odds (say 3 in favor of the alternative). Multiplying these numbers yield the posterior odds (in this case, 3 * 1/3 = 1). Then you can transform these odds to a posterior probability (here, 1/(1+1) = 0.5). We don't generally allow users to input prior model odds but we may add this in the future.
Cheers,
E.J.

JH89

Hi E.J.,
great, thanks for this very quick and understandable answer. However, I do have a follow-up question(s):

I guess the prior odds are computed by p(H0)/p(H1). Are the prior odds then just an effect size (normally in favor of H1) you would expect?

And I was also a bit confused that in JASP you provide prior information for the effect size of H1 (is this denoted as Cohen's d for Bayesian t-tests?) to compute the BF. But is this prior information not the very information you would need to compute the prior odds?

Sorry if may questions just show a lack of understanding.

Cheers,
Jonas

EJ

Hi Jonas,

It is indeed a little confusing, because "prior" means different things. On the level of models, the prior p(H1) means "what is the relative plausibility of H1?"; on the level of parameters within a model it means "what is the relative plausibility of specific effect sizes, given that the model holds? (e.g., given that H1 is true, what effect sizes do I expect)".

For the t-test, prior information on effect size is indeed on Cohen's d (the population version, so true mean mu/true standard deviation sigma). But note that this prior (which JASP lets you specify) is for the effect sizes you expect, given that H1 is true. The prior probability that H1 is true is something else, and this is what you need for the prior model odds.

Cheers,
E.J.

JH89

Hi E.J.,

thanks for pointing these conceptual differences out. I don't want to torture you with questions but maybe a final one

Would it not be reasonable to also base the prior odds on effect sizes you found in the literature? E.g. if I can compute an average Cohen's d = 1 from several findings in the literature should this not be reflected in the prior odds? Of course the problem would be what a Cohen's d of 1 would mean in terms of the prior odds. Could you e.g. assume that H1 is double or three times as likely than H0?

Thanks a lot
Jonas

EJ

Hi Jonas,

In principle, the question of the presence of an effect is independent of the strength/size of that effect. I would argue that the prior odds can still be based on outcomes for earlier experiments, but then in terms of Bayes factors that were obtained earlier. These earlier BFs will generally be higher when Cohen's d is higher, but not always. So if we have n=5 and d=1, I am less confident that the effect exists than with n=5000 and d = 0.5.

Cheers,
E.J.

JH89

Hi E.J.,
perfect, many thanks for your help. Good luck with this and your other projects.

Best
Jonas