How to interpret the post-hoc tests of Bayesian ANOVA?

eniseg2

Hi everyone,
I am struggling with the interpretation of the post-hoc tests in the Bayesian ANOVA.

Do the posterior odds tell me the strength/likelihood of the effect being present with larger posterior odds indicating a stronger effect? Or is this what the BF01,U column is for?

Or do the posterior odds just tell me how much has been learnt from my data in comparison to the prior?

Kind regards,
eniseg2

EJ

The posterior odds are for the effect being present, where the prior odds have been corrected for multiplicity (the post-hoc character of the test). The BF01,U gives the evidence in the data without correcting for multiplicity (the U stands for uncorrected).
Cheers,
E.J.

DMac18

Hi E.J,

Just seeking clarification regarding the interpretation of the BF for post-hoc tests for Bayesian ANOVA. I've seen another of your responses to this similar query, your other response being:

"The posthocness expresses itself through the prior model probabilities. The BF remains the same".

Are you saying that we only need to adjust for multiplicity by correcting the prior odds? In other words, the fact that the BF is uncorrected is essentially inconsequential - since we have already adjusted for multiple tests and can report the BF without fear of overstating the effects?

Some Context:
I have one continuous dependent variable, one continuous covariate, and 3 conditions (3 experimental conditions and a control) and am interested in applying Bayesian framework because I have run the same experiment on 3 separate occasions, and wish to pool the results. I am mostly interested in 4 specific contrasts, but as there is currently no option in JASP for specifying contrasts with Bayesian ANCOVA, (or ANOVA), so I'm running Post-hoc tests, which look promising in any case.

I could perhaps separate the data into 4 separate csv sheets, each only containing the 2 conditions I'm interested in comparing. But with this approach, I would be unsure how to account for multiplicity?

Kind regards,
Dmac

EJ

Yes, the Bayesian account for multiplicity is entirely in the prior model probabilities. To respond to your question:

"Are you saying that we only need to adjust for multiplicity by correcting the prior odds?"

Yes

"In other words, the fact that the BF is uncorrected is essentially inconsequential - since we have already adjusted for multiple tests and can report the BF without fear of overstating the effects?"

If you have multiple tests, and not a clear and unambiguous prior opinion on the post-hoc tests (so they are really post-hoc, not planned) then the correction is through the prior probabilities. But this correction does need to be applied. So simply reporting the BF, without applying the correction, would be ill-advised (unless you are absolutely transparent about what it is you are doing)

Cheers,
E.J.