Bayesian Anova results interpretation
I'm a french psychologist (sorry in advance for my english!!!) and a reviewer for a article wants us to use statistics Bayesian anova! I don’t know anything on this topic and I’m desperately trying to understand bayesian procedure !!! I’ve downloaded JASP (that is actually easy to use), ran the Bayesian repeated measures ANOVA and then… I don’t know how to read the results and how to write it in the manuscript.
For example, in the paragraph below, I would like to discuss about the lack of significance (in red). It is a good thing for me : I wish to show that this insignificant result is consistent with the null hypothesis
In the control group, the interaction of response × orientation, which measures the compatibility effect, was significant (F(1, 17) = 5.255; P = .035; η2 = 0.24), as was the 3-way interaction of prime × response × orientation (F(1, 17) = 6.642; P = .020; η2 = 0.28). For this group, in the congruent prime condition, right-hand responses were faster when the orientation of the object was also to the right (M = 658 ms; SE =23) rather than left (M = 677 ms; SE = 24) (F(1, 17) = 6.582; P = .020; η2 = 0.28). Similarly, left-hand responses were faster when the orientation of the object was also to the left (M = 647 ms; SE = 24) rather than the right (M = 678 ms; SE = 31) (F(1, 17) = 5.802; P = .028; η2 = 0.25). The interaction of response × orientation was significant (F(1, 17) = 9.602; P = .007; η2 = 0.36). By contrast, in the incongruent prime condition, the interaction of response × orientation was not significant (F(1, 17) = 0.358; P = .558; η2 = 0.02) (Figure 2).
“response” (= “main” in JASP below ) x “orientation” (= ”orient” in JASP)
And I found this in JASP:
Model Comparison - dependent
Models P(M) P(M|data) BF M BF 10 % error
Null model (incl. subject) 0.200 0.635 6.951 1.000
Main 0.200 0.155 0.735 0.244 1.485
Orient 0.200 0.158 0.749 0.248 1.324
Main + Orient 0.200 0.038 0.158 0.060 2.344
Main + Orient + Main ✻ Orient 0.200 0.014 0.058 0.023 4.181
Note. All models include subject.
Analysis of Effects - dependent
Effects P(incl) P(incl|data) BF Inclusion
Main 0.600 0.208 0.175
Orient 0.600 0.210 0.177
Main ✻ Orient 0.200 0.014 0.058
What are the important data to use in order to respond to the reviewer? Can I use the result of the interaction : BF10 (Main + Orient + Main ✻ Orient) = 0.058 or I have to divide (Main + Orient + Main ✻ Orient)/( Main + Orient)?
What is the difference with the BF inclusion (Main ✻ Orient)=0.058 ????
Could you help me with this????
Thank you very much in advance for your answers, advices, comments….
Take care
AnneG
Comments
Hi AnneG,
There is a lot of information here. In JASP, are you trying to test the Main * Orient interaction? In that case, the relevant output is:
Main + Orient 0.200 0.038 0.158 0.060 2.344
Main + Orient + Main ✻ Orient 0.200 0.014 0.058 0.023 4.181
BF_10 for the two main effects model is .060, and BF_10 for the model with the interaction is .023 (that is, adding the interaction weakens the BF); consequently, the BF against including the interaction is .060/.023= 2.6.
However, note that the BFs are lower than 1, supporting H0. In fact, the null is supported across the board (BF_10 < 1). This is also evident from the posterior inclusion probabilities for the effects, which are all lower than the prior inclusion probabilities. So I would be hesitant to draw strong conclusions here (if this is the analysis you are planning to do). I am currently writing a tutorial paper on the interpretation of Bayesian ANOVA output, and I also recommend this paper:
http://www.ejwagenmakers.com/inpress/RouderEtAlinpressANOVAPM.pdf
Cheers,
E.J.
Unfortunately, the link to this paper does not work...
Ah, this one then: http://www.ejwagenmakers.com/2017/RouderEtAl2017ANOVAPM.pdf