Correction for multiple testing of post hoc Bayesian t-tests

Comments

• EJ

  October 2020

  Hi Nils,

  It is conceptually cleaner to say that the BF is just the BF -- for an assessment of the evidence does not matter how many hypotheses you are testing. But if you were testing many hypotheses, in a haphazard fashion, this usually indicates that each test by itself involves a low probability H1. So the Bayesian adjustment for multiple testing is to lower the prior odds. This in turn affects the posterior odds, but the change from prior to posterior odds (i.e., the BF) is independent of the number of other hypotheses that are tested.

  Cheers,

  E.J.
• NilsPixa

  October 2020

  Hi E.J.,

  Thanks for your quick reply. Okay, so it is fine to report the provided BF, U in the table of post-hoc t-tests, e.g., to disentangle an interaction effect of a RM-ANOVA with multiple (let's say three) groups?

  Best

  Nils
• EJ

  October 2020

  Well, if you would just report BF_U you would ignore the post-hoc character of the test. So I would report the BF_U but also the posterior probabilities (which are corrected for the post-hoc nature of the test)
• NilsPixa

  October 2020

  Well, yes, that is also a fine recommendation. Maybe that can be inserted in the preprint of van Doorn et al.The JASP Guidelines for Conducting and Reporting a Bayesian Analysis.

  How would you abbreviate or label the posterior odds in a manuscript (to be readerfriendly 😉)?

  Thanks and best

  Nils
• EJ

  October 2020

  Lindley (1985) uses the zero, so "0", but a lower-case "o" seems to be more common (https://en.wikipedia.org/wiki/Odds)
• NilsPixa

  October 2020

  Okay, many thanks E.J.!

  Best

  Nils