Bayes Process need help processing

Michael_Jasper

Bayes Process only gives measures (AIC, WOAIC, etc.) that make sense when compared to something else. I would like to get some measure of null hypothesis to compare with. What do you think about making the same model (same interrelations between same number of variables) but with completely random data, would that satisfy as a null model to compare with?

Why do I want this? I would then have a delta in these output parameters, between alternative and null hypotheses, and using some approximation formula that I've seen around I can calculate a BF10 value.

Michael_Jasper

Instead of using random data I've opted for completely uncorrelated data. So I set up the same model interrelation pattern but with dummy variables that are completely uncorrelated with each other. Each interrelation Pearson precisely equals zero (0). So, this is my null hypothesis model, and any comment on the wisdom (or stupidity) of this is very welcomed and appreciated.

Does anyone know an equation that can turn an AIC value difference (between alternative hypothesis model and null model) into a BF10 value? If possible from a citeable location.

Michael_Jasper

Regarding the above, chatgpt has given me

BF10​=exp(−ΔAIC/2)

But won't tell me where it got that from. @EJ is this yours? If so, do you have any paper I can cite for it? Is it valid (as an approximation of course)?

Michael_Jasper

Sorry, to post-post ammend, meaning BIC instead of AIC.

Michael_Jasper

Hit snag with this in that JASP Bayesian process module won't allow me to make a model with variables that are perfectly correlated, and reports this in an error message. Bit confused as for the null model the variables have a correlation of zero, and so I don't know why it is saying they are perfectly correlated. I logged them to distort them a little and still it doesn't work. Any suggestion(s) what to do now?

Error message in full:

The following problem(s) occurred while running the analysis:

• The variance-covariance matrix of the supplied data is not positive-definite. Please check if variables have many missings observations or are collinear
• Note: The following pair(s) of variables is/are perfectly correlated: LOG Zero FF and LOG Zero HR. Note that if you have specified a weights variable, the correlations are computed for the weighted variables.

Michael_Jasper

Instead of trying to do a null model in JASP, now trying to do it in blavaan, and have got some way with it but not quite there and I have posted to ask for help in the blavaan group here: https://groups.google.com/g/blavaan/c/TbSMIJXsPi0

EJ

A paper on how to go from AIC or BIC to model weights:

Wagenmakers, E.-J., & Farrell, S. (2004). AIC model selection using Akaike weights. Psychonomic Bulletin & Review, 11, 192-196. https://www.ejwagenmakers.com/2004/aic.pdf