JASP Bayesian ANCOVA Model Averaged R2

Mcube78

Hi,

I am struggling a little bit to get some information on the meaning of the "Model Averaged R2" output in a Bayesian ANCOVA (JASP 0.11.1).

The short context is that a reviewer *really really* wants to see R2 and 95% CIs from a frequentist ANCOVA to understand how much extra variance the variables in the winning model (in the bayesian approach) explain over the null or the model without those key variables the BF of the model over the null or the model with all the "other" variables was apparently not persuading despite them being very large in terms of weight of evidence..).

My first thought was to reverse-infer the R2 from the BF, but I guess I can only get a proportionality relation between the two (i.e., I can calculate BF from 2 R2, but cannot go the other direction -- right?). So next, I found the "Model Averaged R2" option, but I'm not sure I fully understand it. Is that the "non-frequentist" R2 (i.e., the R2 considering all the models, not just the point-estimate of the best model a la frequentist R2?)

I ask because unfortunately there is very little on the bayesian ANCOVA -- I spent quite some time on google but with no luck, and the Bayesian ANCOVA has no entry in the https://jasp-stats.org/how-to-use-jasp/

Thank you

Martin

Mcube78

After a few more hours of digging, I did find a post about "single model" statistics for ANOVA, which includes an R2, so I'm going to assume that that's probably the closest to what the reviewer is asking (here: https://forum.cogsci.nl/discussion/5982/95-credibility-intervals-for-bayesian-ancova )

So my question now is how is this R2 calculated --is it something like Gelman et al "R-squared for Bayesian Regression Models"?

https://www.tandfonline.com/doi/full/10.1080/00031305.2018.1549100

EJ

I'll send someone from the team your way...

E.J.

Mcube78

Thank you EJ.

For whomever can answer, I have one more question (again, for my friendly reviewer..)

Is there a way to obtain in the Table of effects (or elsewhere) the model averaged posterior medians and CIs for effects in a Bayesian ANCOVA? I can get generate the BF, P(incl) and P(incl|data), but cannot seem to find a way of getting the individual posterior medians and CIs for the effects. Is there a way?

Thank you for any feedback (and for the wonderful work -- in my lab we <3 JASP!)

Martin

vandenman

Hi Martin,

is that the "non-frequentist" R2 (i.e., the R2 considering all the models, not just the point-estimate of the best model a la frequentist R2?)

Yes. The Bayesian R-squared is derived from Gelman's unpublished manuscript. So R2 is based on the variance of estimated prediction errors. The prediction errors come from the model averaged posterior, so it's not a point estimate of the best model.

Is there a way to obtain in the Table of effects (or elsewhere) the model averaged posterior medians and CIs for effects in a Bayesian ANCOVA?

If you tick "Estimates" it should show the model averaged posterior mean and uncertainty for the effects.

I'm not sure exactly when we implemented this, you may need to update to a newer version than 0.11.1 to obtain this functionality.

Hope that help! If you have any more questions, let me know.

Cheers,

Don

Mcube78

Thank you Don, very helpful!

Cheers

Martin