Bayesian ANOVA - comparing hypotheses
Hi JASP team,
Thanks to JASP, I decided to work for my next article with Bayesian statistics, and it seems I need some help:
My dataset consists of one continuous independent variable (IV) and a categorical variable with four levels (so M1, M2, M3 and M4, the averages of the groups in the IV). If I understand correctly, in a Bayesian ANOVA, JASP will compare HO: M1 = M2 = M3 = M4 with H1: M1, M2, M3, M4. Is that so?
However, based on the literature, my alternative hypothesis would be H1' = M1 > {M2, M3, M4}. Can the Bayesian ANOVA compare H0 and H1'? How do I specify the H1' so that the analysis compares H0 with H1' and not H1?
In the literature, many studies prefer to group M2, M3 and M4 and compare the resulting group with M1. In older literature, researchers were grouping M1, M3 and M4 and comparing the resulting group with M2. Using a Bayesian analysis, I would like to see which of the two groupings is more informative. This would mean the comparison of two hypotheses:
H2 = M1 > {M2, M3, M4} and H3 = M2 < {M1, M3, M4}
Is a Bayesian t-test of a Bayesian ANOVA better suited to compare H2 and H3? If the answer is an ANOVA, how do I define the direction of difference to be tested? (in the t-test, I should be using a one-sided t-test)
Thank you in advance!
Georgios.
Comments
What you want is to order-restrict your H1 model. This can be done in one of the following:
BayesFactor(the Bayesian modeling package used in the internals of JASP).bayestestR'sbayesfactor_restrictfunction, also in R (right now only works withbrmsorrsranarmmodels). Here's a short explanation >>Thanks! (timely and helpful, as always!)
I followed the first suggestion (dus, BAIN in JASP). If I understand correctly, the comparison of the support that each hypothesis receives is given in the Bayes Factor matrix, right?
I am not sure how to read this matrix. In the attached file, does 31735.222 show how much more likely H2 is in comparison to H1, or does it show how much more likely H1 is in comparison to H2? (I would imagine the first, since the H1 is the null hypothesis, but I don't want to rely on my imagination...)
It would see that the matrix is organized with row as the numerator and column as the denominator (@EJ this does seem counter-intuitive...), so 31735.222 represents how much more likely H1 is compared to H2.
You can also see that the PMP (Posterior model probability) is very low for H2. (Also note that the unconstrained model is the most supported model in your analysis...)
I'll pass this comment along
Not sure if this will be of any help (I can also provide you the dataset, if need be), but (I think that) the JASP "normal" Bayesian ANOVA and the BAIN ANOVA give different results (not sure about the specifications, but the overall conclusion would be totally different) - see attachment
(sorry for the trouble, thanks for the help!)
Yes, they operate under different assumptions, so will give different answers. In general, Bain is particularly well suited for testing order-restrictions.
E.J.
Do I proceed in writing with the results I sent you, or is there a possibility for JASP failure?
I think it's fine -- I am not a Bain expert so it is a little hard for me to grasp that table, but the standard JASP ANOVA seems fine.
E.J.