Comparison of Bayes Factor for correlations calculated in JASP vs. in R using the BayesMed package
Dear all,
I recently tried out JASP to run some Bayesian analyses. So far, I have been using R for this, and I was interested in whether I would get the same results, especially since JASP is using R internally for its analyses.
Computing the Bayes Factors for t-tests in JASP lead to the same results as the ttestBF function from the BayesFactor R package (I think JASP uses this function internally). However, the results were different for correlations. Specifically, I calculated the correlation between Agreeableness and Openness in the Big5 example dataset provided by JASP.
Using JASP Bayesian Correlation Pairs to calculate the Bayes Factor for the correlation (with default settings, i.e. Hypothesis: correlated, Beta*prior width = 1), leads to a BF10 = 32.64.
Calculating the Bayes Factor in R with the following script:
library(BayesMed)
dataset <- read.csv("Big 5 (Dolan, Oort, Stoel & Wicherts, 2009).csv",sep=" ")
with(dataset,jzs_cor(Agreeableness,Openness,alternative="two.sided"))
leads to a Bayes Factor of 21.01.
I would be very interested in understanding where the different results come from. It would be great if someone could point me to an explanation. Thanks a lot in advance!
Best regards,
Pascal
Comments
Hi Pascal,
JASP uses the correlation test as originally proposed by Jeffreys in 1961. It assumes a bivariate normal distribution and puts (by default) a uniform prior on rho. The philosophy behind the test is explained here (http://www.ejwagenmakers.com/inpress/JeffreysToPTests.pdf) and the mathematics are here (http://arxiv.org/abs/1510.01188). This is all work by my awesome student Alexander Ly. An application is here (http://www.ejwagenmakers.com/submitted/BayesianBathingHabits.pdf).
I think BayesFactor uses a JZS test (my former student Ruud Wetzels proposed it in PBR: http://www.ejwagenmakers.com/2012/WetzelsWagenmakers2012.pdf). Right, Richard?
Which test you use is a matter of taste and conviction. I am reluctant to argue strongly for one or the other. The Jeffreys test is conceptually simple and straightforward to explain, but the JZS test is consistent with the other tests in the packages. I would see this as an opportunity for a robustness analysis -- most likely the results will point in the same direction.
In a perfect world, JASP would also offer the JZS option, and the BayesFactor package would also offers the Jeffreys option. I think we might want to work towards that goal.
Cheers,
E.J.
Hi EJ,
thanks a lot for your helpful answer! I will have a look at the papers you pointed me to.
In the project I am currently working on, I am also reporting reporting JZS Bayes factors for t-tests. So I will probably report the Bayes factors from the BayesMed package for the correlations - for reasons of consistency.
I also think that it would be interesting to see how both types of Bayes factors relate to each other. I have not had the time so far to do a thorough simulation for that. Instead, I just calculated the Bayes factors for all pairwise correlations in the JASP Big 5 example dataset (see Table and Figure below). As you suggested, the Bayes factors point in the same direction.
Best,
Pascal
EJ,
Didn't Pascal use BayesMed and not BayesFactor to do the JZS correlations? In which case shouldn't you be considering adding the Jeffereys option to your own R package as it only does JZS and JZS Savage-Dickey based correlations?
BayesFactor only does Regression models not as far as I can see correlations directly although I am happy to be corrected on this. Similarly BayesFactor I think
Thanks.
Bryan.
Hi Bryan,
Ah, BayesMed...I will leave that one up to Michele! :-)
Cheers,
E.J.