How to set the prior for correlation matrix with very large data sets
This is maybe more of a theoretical question.
A colleague has a data set from a very large questionnaire survey with over 18,000 participants.
If I understand correctly, the default priors in JASP assume effect sizes typically found in typical psychology experiments with N < 100?
She has created a correlation with a few variables, and of course the BFs are all extremely large, even for very weak correlations. For example, using a stretched beta prior width = 1:
r = 0.095, BF10 = 4.961e +33
r = 0.43, BF10 = infinity.
The data completely overrides the prior.
Are there any methods for defining a prior for such large data sets, which would give the null a chance, and allow hypothesis testing?
Comments
I don't think this is a problem. The default priors are mostly based on general desiderata, "objective-Bayes" style. For the correlation, even if the alternative hypothesis is specified to predict correlations closer to zero (this can be done by lowering the prior width) the evidence would still support the existence of an effect. So the BF shows strong evidence in favor of an effect, which then licenses the estimation process where you can report the credible interval on that correlation. Typically, you'll conclude that the correlation is different from zero but modest in size. If you really want to give the null a chance, you could define an interval null, or a peri-null. In JASP you can test peri-nulls via transitivity (it's a little work): https://jasp-stats.org/2017/10/25/test-interval-null-hypotheses-jasp/
E.J.