Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

BF Hypothesis Testing Question

Hi,

I am writing for the first time a paper within the Bayes framework, and I feel rather comfortable now up to the level of Linear Regression. However, i do have a question regarding how to test the following hypothesis, and if i need to rephrase it.

H:  The negative relationship between X and A is stronger than the negative relationship between X and B.

(A, B being two dimensions of the same construct). As of now i have conducted Bayes correlations using JASP and the BF for XA is about 34 and the for BF for XB about 4. Would comparing the BFs suffice? or is there a "difference test" that i need to know about.

Moreover, I am conducting about 20 correlation tests at once (we are dealing with multiple characteristics), Do i have to worry about multiple testing, and should I specify this in the paper?

Greetings, from the Big Bang,

Georges Lemaitre

Comments

  • Hi George,

    1. About multiplicity: yes, I do think a correction is in order, unless you want to go fully subjective Bayes -- with carefully specified prior plausibilities for all hypotheses involved, there is no need for a correction. Some relevant references and remarks here: https://psyarxiv.com/s56mk
    2. If you want to test whether the correlation between X and A is larger then between X and B, you have two complications: (a) X features in both correlations, so they are not independent; (b) in order to address your question you need a test for the difference in correlation -- simply comparing the BFs is not correct, for the same reason that comparing p-values is not correct (http://www.ejwagenmakers.com/2011/NieuwenhuisEtAl2011.pdf)

    Cheers,

    E.J.

  • Hi E.J.,

    Thanks for the input, will do.

    Just another question, and i will go. What are good places to get feedback on our methods section? The issue at hand is that while NHST experts are everywhere (myself included), i am feeling quite uneasy jumping directly into peer-review with interpretations of Bayesian methods that I am only learning.

    Greetz,

    Georges

  • Hi Georges,

    Well you could post it here, or email some people for feedback.

    Cheers,

    E.J.

  • edited December 2

    Good morning,

    thanks. What I will probably do, is post the preprint on PsyArXiv and link it here for the good people here to look over the methods... It'll take some time though :)

    Have a good day!

    Georges

    PS: Jasp has been godsent for all trying to learn about Bayesian inference. Even old dogs like me. I am very curious however about the reception it will have with the reviewers :) Especially in organizational psych, where bayesian methods are not really present :)


    PSS: having hypothesis stating the H0 is absolutely new and awesome to me ...

  • edited December 2

    Hello George,

    I think the new R package BFpack can do what you are looking for, see section 2.2 in the preprint https://arxiv.org/abs/1911.07728v1

    If you are into R: I used this code to compare pairs of correlations:

    library(polycor) # install.packages("polycor")
    library(BFpack) # install.packages("BFpack")
    
    # compute some correlations using data from the package:
    cors <- hetcor(memory[memory$Group == "HC", -ncol(memory)])
    
    # check out correlations:
    cors$correlations
    
    # compute Bayes factor to compare two correlations that seem to be rather different
    BF(
     cors,
     hypothesis = "Del_with_Im = Fas_with_Del"
    )
    
    # compute Bayes factor to compare two correlations that are similar
    BF(
      cors,
      hypothesis = "Cat_with_Wmn = Fas_with_Wmn"
    )
    
    

    Note that the BFpack package does not return the Bayes factor, but posterior probabilities (based on prior odds specified using the prior argument; the default is 1:1), which I found quite odd, but well, you can compute the Bayes factor from the posterior probabilities.

    Note that my code does not come with any warranty, however. I checked out the package last week and only now tried to use it to compare correlations. You should definitely verify it; the documentation was not 100% clear to me.

    Cheers

    Martin

  • Hi Papi (lol)

    thank you for the code and insight, will run it the first thing tomorrow! Have a good evening...

    Cheers back!

    Georges

Sign In or Register to comment.