BF Hypothesis Testing Question

Comments

• EJ

  December 2019

  Hi George,

  1. About multiplicity: yes, I do think a correction is in order, unless you want to go fully subjective Bayes -- with carefully specified prior plausibilities for all hypotheses involved, there is no need for a correction. Some relevant references and remarks here: https://psyarxiv.com/s56mk
  2. If you want to test whether the correlation between X and A is larger then between X and B, you have two complications: (a) X features in both correlations, so they are not independent; (b) in order to address your question you need a test for the difference in correlation -- simply comparing the BFs is not correct, for the same reason that comparing p-values is not correct (http://www.ejwagenmakers.com/2011/NieuwenhuisEtAl2011.pdf)

  Cheers,

  E.J.
• GeorgesLemaitre

  December 2019

  Hi E.J.,

  Thanks for the input, will do.

  Just another question, and i will go. What are good places to get feedback on our methods section? The issue at hand is that while NHST experts are everywhere (myself included), i am feeling quite uneasy jumping directly into peer-review with interpretations of Bayesian methods that I am only learning.

  Greetz,

  Georges
• EJ

  December 2019

  Hi Georges,

  Well you could post it here, or email some people for feedback.

  Cheers,

  E.J.
• GeorgesLemaitre

  December 2019 edited December 2019

  Good morning,

  thanks. What I will probably do, is post the preprint on PsyArXiv and link it here for the good people here to look over the methods... It'll take some time though :)

  Have a good day!

  Georges

  PS: Jasp has been godsent for all trying to learn about Bayesian inference. Even old dogs like me. I am very curious however about the reception it will have with the reviewers :) Especially in organizational psych, where bayesian methods are not really present :)

  
  

  PSS: having hypothesis stating the H0 is absolutely new and awesome to me ...
• Papi

  December 2019 edited December 2019

  Hello George,

  I think the new R package BFpack can do what you are looking for, see section 2.2 in the preprint https://arxiv.org/abs/1911.07728v1

  If you are into R: I used this code to compare pairs of correlations:

  library(polycor) # install.packages("polycor")
  library(BFpack) # install.packages("BFpack")
  
  # compute some correlations using data from the package:
  cors <- hetcor(memory[memory$Group == "HC", -ncol(memory)])
  
  # check out correlations:
  cors$correlations
  
  # compute Bayes factor to compare two correlations that seem to be rather different
  BF(
   cors,
   hypothesis = "Del_with_Im = Fas_with_Del"
  )
  
  # compute Bayes factor to compare two correlations that are similar
  BF(
    cors,
    hypothesis = "Cat_with_Wmn = Fas_with_Wmn"
  )
  
  

  Note that the BFpack package does not return the Bayes factor, but posterior probabilities (based on prior odds specified using the prior argument; the default is 1:1), which I found quite odd, but well, you can compute the Bayes factor from the posterior probabilities.

  Note that my code does not come with any warranty, however. I checked out the package last week and only now tried to use it to compare correlations. You should definitely verify it; the documentation was not 100% clear to me.

  Cheers

  Martin
• GeorgesLemaitre

  December 2019

  Hi Papi (lol)

  thank you for the code and insight, will run it the first thing tomorrow! Have a good evening...

  Cheers back!

  Georges