Specifying H0 and H1 in Jasp

tamara

Apologies in advance if this is an obvious question, I am just trying to get my head around Bayesian methods and JASP.

Using Bayesian methods in JASP we aim to compare how well our data fit some specified alternative hypothesis, relative to the null hypothesis of "no effect."

However, in JASP I am unclear as to how we set up these two competing hypotheses, or where these competing hypotheses may be visualized. Is it the case that the Prior ('Cauchy prior width') relates to the null of no effect, and the 'hypothesis tab' relates to the alternative...? Or is it simply that we infer the presence of an alternative hypothesis (i.e. effect) via the null ('no effect), although this nature of this alternative is unspecified?

Thanks

sebastiaan

Hi Tamara,

That's a good question. My understanding (possibly incorrect or incomplete) is the following:

• The null hypothesis (H0) is simply that the effect size is 0.
• The alternative hypothesis (H1) is derived from the variance in the data. Say that you measure response times between two conditions, and the RT effect has a standard deviation of about 100 ms. In that case, a mean RT effect of around 50 ms would weigh in favor H1, but a mean effect of around 1 ms would weigh in favor of H0.
• The cauchy prior determines what effect sizes are considered realistic under H1. This is a distribution, not one specific value. Higher cauchy priors weigh in favor of large effects, so they are generally conservative, tending toward H0.
• The default cauchy prior of 0.707 is chosen so that H1 includes realistic effect sizes, based on the rule of thumb that mean effect sizes tend to be about half of the standard deviation of the effect.

Long story short: Using this Bayesian t test, you don't explicitly specify H1, because it is derived from the data based on rule-of-thumb logic. There are other Bayesian t tests in which you do explicitly state H1, but these are not (yet) implemented in JASP.

I read this somewhere in a nice blog, but I cannot find it anymore. Can anyone confirm (or disconfirm) my understanding of this Bayesian t test?

Cheers,
Sebastiaan

EJ

Hi Tamara,

In most Bayes factor tests, all that matters is the prior on the parameter of interest. For the t-test, for instance, the parameter of interest is "effect size $\delta$". In the usual situation, there are two hypotheses: H0, which stipulates that effect size is zero; and H1, which relaxes that assumption and assigns $\delta$ a prior distribution.

In JASP, you do not need to specify H0 -- this has been done for you automatically. There is not much to specify anyway: H0 simply says that $\delta = 0$. However, you do need to specify a prior for $\delta$ under H1. This happens under "Prior ('Cauchy Prior Width')". When you tinker with that setting and tick the plot option for "prior and posterior" you will see that this setting affects the width of the prior distribution (under H1).

The "hypothesis" options allow you to add more information about the direction of the effect. Again, tick the plot options and check it out!

Cheers,
E.J.

EJ

Hi Sebastiaan,

Good points. However, I would not say that the prior is obtained/derived from the (observed!) data. As the name suggests, the prior is specified before the data are observed, and ideally it reflects our expectations about the size of the effect, should it be present. These expectations have been shaped in part through earlier data, of course, so in that sense I agree.

Also, in my thinking the specification r~Cauchy(.707) explicitly defines H1. When you say "explicitly specifies", perhaps you are thinking of a point such as r = .40? This specification is almost always unrealistic, because there is always uncertainty about the true r under H1. But it may nevertheless be useful and you are right that JASP does not do this test (yet!).

Cheers,
E.J.

sebastiaan

When you say "explicitly specifies", perhaps you are thinking of a point such as r = .40? This specification is almost always unrealistic, because there is always uncertainty about the true r under H1.

Exactly. So to avoid confusion: Would you agree that it's fair to say that you can characterize H1 (in this test) as follows?

• H1 is not specified as an actual value, or range of values. (This, to me, would be an explicit prediction—realistic or no. For example, if I would do a Posner cuing paradigm, I would expect a cuing effect of around 20 ms.)
• Rather, H1 is specified as a range of effect sizes relative to the variance in the data (like Cohen's d). In that sense, the actual values that comprise H1 are derived from the data. But how they are derived is specified in advance based on prior experience and common sense.
• H1 is not derived from the data in the sense of post-hoc cherry-picking.

tamara

Thanks so much for your replies. Incidentally, I found this paper, which also helped my understanding quite a bit. (I'll leave here for others).

http://www.ncbi.nlm.nih.gov/pubmed/21302025