BF and Cred.Interval seem to conflict ?

Comments

• EJ

  March 2016 edited March 2016

  Here we go:
  1. "H0: score =< 6.4 H1: score > 6.4" This is the classical test. I have always considered it illogical, for the following reason. We have the two-sided test that compares H0: score = 6.4 against H1: score \neq 6.4. Now we have the expectation that the effect, if it exists, is really > 6.4. In response, the classical test then goes and changes....the null hypothesis! Weird. In the Bayesian version, what changes is the prior parameter distribution under H1, so that all mass is assigned to values greater than 6.4. This is also apparent from the prior and posterior plot.

  You say: "The 95% Credible Interval is 0.015 ....0.27. I read this as: the value is between (6.4 + 0.015) and (6.4 + 0.27) = 6.415 ...6.67". Yes, this is correct. Note that this is a central credible interval, not a highest posterior density interval. Also note that this is the interval conditional on H+ being true.

  2. H0 is always the absence of an effect. H1 is specified in the input panel; it is also visualized in the prior-posterior plot. The one-sided specification simply "folds" the prior around 0, assigning all mass to values consistent with the proposed direction of the effect.

  3. Yes there can be an apparent conflict between the credible interval and the BF. Note that the credible interval is conditional on H+ being true. The data suggest that H+ may not be true (although the data are ambiguous). In general, the credible interval does not provide information about H0. It is tempting to think it does, but that's wrong. For enlightenment, see for instance:

  http://www.ejwagenmakers.com/inpress/MarsmanWagenmakersOneSidedPValue.pdf

  https://osf.io/m6bi8/

  Cheers,
  E.J.
• pieter

  March 2016 edited March 2016

  Super again. Will read those papers too.
• pieter

  March 2016 edited 1:12PM

  I am confused. Wouldn't it be clear to show

  • TWO priors (corresponding to H0 and H+)
  • the data
  • TWO posteriors

  What I now see in JASP, is:

  • a single prior, which I assume belongs to H0
  • a single prior, which (as EJ tells) depends on H+

  My question: what exactly is depicted in the Bayesian T-Test plots?
• EJ

  March 2016 edited 1:12PM

  Hi Pieter,
  1. For the parameter under test (such as effect size), H0 specifies a point (e.g., delta=0), not a distribution. So under H0 we have a spike at 0, both as a prior and as a posterior.
  2. Therefore, the prior and the posterior you see in the plot belong to H1 (or H+, for a one-sided test). The dotted line is the prior, the solid line is the posterior.
  E.J.
• pieter

  March 2016 edited 1:12PM

  EJ, what I am ultimately interested in, are posterior odds. That is, how likely are H0 and H+, given the data.

  Of course I'd need to enter prior odds. Also, I'd like to define a more realistic H0, that is, not a spike but a distribution.

  How would you do this in JASP? Or would I have to go to R ?
• EJ

  March 2016 edited 1:12PM

  Hi Pieter,

  Entering prior odds is something you can do easily yourself; multiplying them with the Bayes factor gives the posterior odds. If you want H0 to be a distribution, then you'll have to look at some of Richard's work on interval Bayes factors. We might implement this in a future JASP release, but before we do so we'll probably allow the user to be more flexible with respect to the expectations under H1 instead.

  Cheers,
  E.J.