How does BayesFactor account for random effects?
If I understand correctly from Rouder et al's (2012) paper, the way BayesFactor "deals" with random effects is:
- Set a wide prior (
r = 1) - No sum-to-zero constraint.
But I see no mention of how the presence of random effect affects the computation of the likelihood of related fixed effects (i.e., is there any difference when an effect is between-subjects vs within-subject).
The use of random factors as random effects vs fixed effects with a wide prior in BayesFactor seems to have little effect:
library(BayesFactor) data(md_12.1, package = "afex") # BayesFactor - specify "id" as a fixed effect. m0_f_BF <- lmBF(rt ~ id, md_12.1, rscaleEffects = c(id = 1)) m1_f_BF <- lmBF(rt ~ angle + id, md_12.1, rscaleEffects = c(id = 1)) BF_f_BF <- unname(as.vector(m1_f_BF / m0_f_BF)) # BayesFactor - specify "id" as a fixed effect. m0_r_BF <- lmBF(rt ~ id, md_12.1, whichRandom = "id") m1_r_BF <- lmBF(rt ~ angle + id, md_12.1, whichRandom = "id") BF_r_BF <- unname(as.vector(m1_r_BF / m0_r_BF)) c(as_fixed = BF_f_BF, as_random = BF_r_BF) #> as_fixed as_random #> 909.1889 900.1979
However the differences are much larger with other methods (below I use the BIC approx. for simplicity, but stan-based methods also produce differences that BayesFactor does not):
library(lmerTest)
BIC_BF <- function(m0,m1){
d <- (BIC(m0) - BIC(m1)) / 2
exp(d)
}
# BIC approx - specify "id" as a fixed effect.
m0_f_lm <- lm(rt ~ id, md_12.1)
m1_f_lm <- lm(rt ~ angle + id, md_12.1)
BF_f_lm <- BIC_BF(m0_f_lm, m1_f_lm)
# BIC approx - specify "id" as a random effect.
m0_r_lm <- lmer(rt ~ (1|id), md_12.1)
m1_r_lm <- lmer(rt ~ angle + (1|id), md_12.1)
BF_r_lm <- BIC_BF(m0_r_lm, m1_r_lm)
c(as_fixed = BF_f_lm, as_random = BF_r_lm)
#> as_fixed as_random
#> 5281.736 3920528.548
Might this be the root of the somewhat common question here in the forum regarding differences between frequentist and Bayesian rmANOVAs in JASP?
Comments
I'll forward this to Richard...
Hi @EJ, any news on this?
No. I'll ping them again.
E.J.
Thanks.
I've since reached out to Jeff Rouder - I will update here if I hear from him.
Thanks again and happy holidays!
Jeff has addressed this issue in this recent presentation: https://www.youtube.com/watch?v=PzHcwS3xbZ8
Thanks for the link!