Welcome!

Sign in with your CogSci, Facebook, Google, or Twitter account.

Or register to create a new account.

We'll use your information only for signing in to this forum.

Supported by

Setting Priors in JASP

MSBMSB Posts: 25
edited June 2016 in JASP & BayesFactor

Hi all,

From what I understand, one can set the Cauchy prior width acordding to a speculated effect size when preforming a Bayesian t-test, such that if I think my effect should be around 0.6, I would set the Cauchy prior width to 0.6 (correct me if I'm wrong).

What I still don't understand is the Beta* prior width for correlations - can it be used in the same manner?
If I expect an r=~0.6, would I set the Beta* prior width to 0.6?

Or in other words, how do I translate an expected effect size/correlation to Beta* prior width?

Thanks,
Mattan

Comments

  • EJEJ Posts: 392
    edited 7:55PM

    Hi MSB,

    The best way to understand the shape of the prior is to inspect the plots. A Cauchy prior with of 0.6 still has the center of the mass at zero. When specified as a 2-sided test, this means that 50% of prior mass falls between -0.6 and +0.6. This may sound rather wide but keep in mind that the prior is centered on zero, assigning most mass to values very close to zero. We are working on modules that allow one to specify priors with a location parameter away from zero. Until that time, I recommend to set the width at the default value (at least). Making it smaller causes H1 to behave like H0, leading to an uninformative test.

    For the correlation test, parameter rho ranges from -1 to 1. So we take a Beta(a,a) prior (which ranges from 0 to 1), we stretch it so that it ranges from -1 to 1 (this is the Beta* distribution) and then the prior width is given by 1/a. So when you set the prior width to 1/2, you should recognize the shape of a Beta(2,2) distribution.

    Cheers,
    E.J.

  • MSBMSB Posts: 25
    edited 7:55PM

    If I understand you correctly,

    If I have a one-tailed hypothesis that rho is (-0.4), I would set the beta* prior to 0.4*2=0.8 and test for a negative correlation - this would give me a distribution with 50% of prior mass between 0.0-(-0.4) and 50% of prior mass between (-0.4)-(-1.0).
    Right?

    As for t-tests, unfortunately sometimes the default prior is just too wide for some smaller effect sizes (robustly small :-( ).

  • EJEJ Posts: 392
    edited 7:55PM

    Hi MSB,

    (1) The interpretation I gave (50% mass between -x and x) only holds for the Cauchy prior, which is used for the t-test. It does not hold for the beta-star prior, which is used for the correlation test. It is best to just inspect the prior distribution visually, which is relatively straightforward in case of rho.
    (2) If you really want to test H0: rho=0 against H1: rho=-0.4, then you want a likelihood ratio. The question then becomes, how can you be so sure about the value of -0.4? Anyway, I expect new versions of JASP to include this kind of likelihood ratio testing as well.
    (3) For your correlation problem, I would simply use the uniform prior and do a one-sided test, potentially exploring robustness by changing the prior width somewhat.
    (4) I cannot stress this enough: if you want to have the alternative hypothesis predict predominantly small effect sizes (so have Cauchy prior width equal to 0.3 or something) you also have to shift the location parameter away from zero. Otherwise the models just become indistinguishable. So there are two opposing forces: (I) a large width can work against H1 when the true effect is relatively small; (II) centering the prior on zero works in favor of H1 when H0 is true. Ideally the user can change both -- this is in the works but not implemented yet. Until it is implemented, my advice is not to stray too much from the default settings. It is fine to go subjective but it can be dangerous to stop halfway.

    Cheers,
    E.J.

  • MSBMSB Posts: 25
    edited 7:55PM

    Hi EJ,

    I sure do like the subjective part of Bayesian inference - but I think I understand your point.
    Looking forward to all the awesome things coming out in future releases.
    Until then - thanks again!

    M

  • I have a similar question. I'm looking at a Bayes ANCOVA in JASP. I want to try and make sure I'm including as much of an informed prior is possible for my model comparison. I'm testing whether or not a certain level of training results in higher levels of certain practices. There's some prior research to suggest the effect of this training is in the area of .20 - .40 d. So it's greater than a null or uninformed prior. How do I incorporate this prior information into a Bayesian ANCOVA prior in a JASP ANCOVA?

  • EJEJ Posts: 392

    Hi Scott,
    For the t-tests, JASP now has "informed priors" that allow great flexibility in their specification. In particular, they can be centered away from zero (https://arxiv.org/abs/1704.02479). We are working to make the ANOVA and ANCOVA similarly flexible, but this will take some time. I am not 100% sure whether the BayesFactor package in R allows you to implement an order-restriction; in my experience, adding the order-restriction (here: training helps, and does not hurt performance) and setting the prior width together provide a decent approximation to what you would get with a more informed distribution that is not centered on zero.
    Cheers,
    E.J.

  • larrylarry Posts: 3

    Hi EJ,
    I just wanted to check with new informed priors function in JASP. When you use the "normal" section is the "mean" a standardised effect size (Cohen's d), and is the "std" the variability around this effect size (d)?
    Also for the "t" section of the informed priors, I have read that the "Oosterwijk Prior" of location 0.350, scale 0.102, and 3 degrees of freedom, is a good model for a small-medium effect. Did you have any other suggestions for a medium-large effect and a large effect?
    thanks for any help,
    love the program
    Larry

  • EJEJ Posts: 392

    Hi Larry,
    1. Yes, the priors are all on delta, the standardized effect size (and sd indicates the uncertainty)
    2. We are in the process of eliciting a number of other informed prior distributions.
    3. For medium-large and large: interesting question. Of course, with large effects the test becomes redundant (it will be apparent from the data). My first thought is to take the Oosterwijk prior and just shift it. Maybe to 0.650 for medium-large, and 0.950 for large?
    Cheers,
    E.J.

  • larrylarry Posts: 3

    Thanks EJ,
    Really helpful
    cant wait for more updates
    Larry

Sign In or Register to comment.