Interpreting moderation effect in Bayesian ANCOVA output
Hi all!
I'm using JASP for a Bayesian ANCOVA where I'm interested in a moderation effect, and am wondering how to interpret the output.
Here's what it looks like:
So my dependent variable is involvement ("T2_apq_subbetrokken") at posttest in an intervention study.
I have two conditions ("conditie": control/intervention) and want to know if there is a main effect of condition. Additionally I want to know if the effect of condition is moderated by PTSD severity at baseline ("T0_PCL_cati": low, medium, high).
I controlled for baseline level of involvement ("T0_apq_subbetrokken") by adding it as a covariate. To make interpretation easier, I added this to the null model, and checked "compare to null model".
What I'm wondering now is how to report the evidence specifically for the moderation effect. Looking at this blog https://www.cogsci.nl/blog/interpreting-bayesian-repeated-measures-in-jasp I would think the BF10 for the moderation effect is (0.342 / 0.191 = 1.791). But that would mean evidence for the moderation effect (albeit little) whereas both BF10 and BFincl show evidence against it. Is it wrong in this case to calculate the moderation effect BF10 like this? What should I report?
Any guidance would be much appreciated, thank you!
Comments
Sorry for the tardy response. It is important to check that your covariate does not overlap (substatially) with PTSD severity, or you will be removing the very thing you are interested in.
As far as the results go, the null model predicts the data best, and the inclusion BFs show weak evidence against including any of the factors. I see why you would want to test for the presence of the interaction by comparing it to the two-main effect model, but in this case that two-main effect model is outpredicted by the null. Either way, none of these BFs are compelling, so it looks more like a case of "absence of evidence".
E.J.
No problem at all E.J., thank you for your response. Regarding the overlap between the covariates and PTSD severity, I think that should be OK. They are not too strongly correlated.
I see your point on absence of evidence, and will report it like that. I actually have a similar case on another endpoint, but where the interaction effect BF10 is a bit larger (0.862/0.125= 6.896 so in the moderate range). BFincl for the interaction effect is only anecdotal though (1.830). Would it be right to interpret these findings like: the null model was better than the other models, so despite there being moderate evidence for the interaction in isolation from the main effects, overall there is not really any evidence for or against our moderation hypothesis?
Yes, but there is another possibility. It may be that the data provide evidence for a "pure" interaction -- so there is only the interaction, and no main effects. This violates the "principle of marginality" which states that if an interaction is part of the model, so should the constituent main effects. However, based on your results I suspect that the best model may be the one with only the interaction. Under advanced options there is the possibility of removing the restriction that the constituent main effects are always along for the ride -- you might want to check that out.
Cheers,
E.J.