Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

ESs and Credible Intervals for One vs Two Sided Bayesian t-Tests

Dear colleagues.

I’m looking for a bit of advice on reporting ES delta and its 95% credible interval in the context of a one-sided Bayesian t-test computed in JASP.

The figure below is the posterior distribution of ES delta under a default one sided prior distribution. A one sided prior was used as the relevant hypothesis was theoretically derived and directional.The BF is non-diagnostic. This is fine (as they say, data don't care what your hypotheses are), and not the issue that I'm concerned about. The median of the posterior distribution is .17, and its 95% credible interval grazes zero (but can’t extend below it).

Should I report the above ES and the corresponding credible interval?

Or should I re-run the analyses with a two sided prior (see below), ignore the BF, but report ES delta = .10, 95% Credible Interval [-.27, .47]? If so, how should I explain this in the manuscript?

The latter (report and interpret the one sided BF followed by two sided ES and credible interval) appears to be the approach that Wagenmakers et al. (2016) have taken here, in a context which is pretty similar to the one that I am in currently: https://frontiersin.org/articles/10.3389/fpsyg.2015.00494/full. However, this seems inconsistent to me for reasons I can’t articulate. Is anyone able to explain to me why this is the sensible thing to do (or not)?

Thanks in advance for any and all advice received. I’m quite new to this Bayesian business, and still very much a novice. Peter Allen

Comments

  • Dear Peter,

    Sorry for the tardy reply. Let me explain my reasoning. In early work, we would specify a directional hypothesis and use that both for BFs and for effect sizes. This is consistent and sensible. So why have we started to use a two-sided prior for ES? There are several reasons but one of them is that for a one-sided ES people might say: "ah, the effect is small but the central 95% interval does not overlap with zero". Of course this has to happen because of the restriction, but this is what people will quickly forget. So for ES we use the two-sided analysis (which is also similar to the classical way of doing things) but for the BF, which depends on predictive adequacy, we do respect the directional nature of the prior. But I completely accept that there are different ways of doing this.

    Cheers,
    E.J.

  • Hi E.J. No problem at all! Now that I've had time to think about it, your approach does make a lot of sense, and it is one that I will be adopting in my research. Thanks again. Peter Allen

Sign In or Register to comment.