Algorithms for estimating BF of ANOVA in JASP/BayesFactor

WangYH

Hi JASP team @andersony3k @booradley @eduard @EJ @EJ @elisa ,

I have seriously read the papers about how to estimate BF, which were written by EJ, Morey, Rouder, Lee and other famous scholars. These articles have enlightened me.

However, I am still confused by the algorithms for estimating BF in JASP/BayesFactor. There are several methods to estimate BF, like basic marginal likelihood identity, Savage–Dickey method, transdimensional MCMC (encompassing Reversible jump MCMC and product space method).

Theoretically, BF equals to the marginal likelihood (ML) ratios of two competing models. From my own perspective,

the logic of basic marginal likelihood is that: first, using MCMC (Gibbs or MH) to estimate the parameter posterior; second, according to Bayes rule, we can get ML simply exchanging the role of calculated parameter posterior and ML in equation. Therefore, the adjustment of parameter prior can affect BF, but model prior does not.

I think transdimensional MCMC is more direct because it estimates BF on the model aspect, not parameter. We can directly obtain model posterior ratios after transdimensional MCMC, then divided by model prior ratios, which in turn obtaining BF. Consequently, both model prior and parameter prior influence BF.

I am not sure whether my thoughts for these algorithms are true. Please reply to me at your convenience.

Best wishes,

Wang

EJ

Dear Wang,

Exchanging the role of the posterior with the marginal likelihood yields the Chib estimator, if I recall correctly (there is also a Chib & Jeliazkov paper). In transdimensional MCMC, the model prior matters for the convergence, but ultimately the BF remains a ratio of marginal likelihoods and is therefore independent of the prior model probability. So although one may give different interpretations to the BF (e.g., the change from prior ro posterior odds; the ratio of predictive performance) in the end it remains the same quantity.

Cheers,

E.J.