moderation analysis in JASP using bayesian multiple regression

lemakru

hi everyone! :)

I am currently writing my bachelor thesis and I am supposed to do a moderation analysis using bayesian multiple regression. Unfortunately, we never learned anything about bayesian statistics so I'm left with quite some questions. I informed myself how to interpret bayesian multiple regression but I'm not sure how to interpret it in the sense of a moderation anaysis. I hope, someone might have some answers for me, so here are my questions:

1. Say, in the regression analysis I use variable X, variable Y and the interaction X*Y as predictors for my response variable Z. The posterior summary of coefficients of one analysis reveals very strong evidence for variable Y (BFInclusion > 30) and strong evidence for the interaction X*Y (BFInclusion > 10). There is no evidence for variable X. The best model in comparison to the null modell is the model including Y and X*Y. Can I speak of a moderating effect of the interaction in this case? Or does either 1) the BFInclusion of the interaction has to be the highest and/or 2) the regression model only including the interaction X*Y be the best model in order to speak of a moderating effect?
2. I have to conduct another analysis using different predictors (but also checking for moderating effects). The analysis reveals that there is no evidence for all my predicting variables (including the interaction) and that the null model is the best model for predicting my response variable. How do I report the results in this case? In case a model including one (or more) predictor(s) is the best one, I would normally report the following: "There is moderate/strong/... evidence for the model including variable A and B (BF10 = ...) in comparison to the null modell." Now, as the null model is the best prediction model, I'm unsure which model I should compare the null model to (as in total there are seven other models with different combinations of predictors) and if I have to report the Bayes Factor or not.

I would really appreciate any help. Thank you a lot in advance!

EJ

Dear lemakuru,

1. I think that in order to test the moderation, the focus is on the interaction term (e.g., https://jasp-stats.org/2020/03/12/mediation-and-moderation-analysis-in-jasp/) There are various model comparisons that are relevant here (e.g., the model without the interaction term vs. the model with the interaction term comes to mind in particular).
2. If you have no evidence for the inclusion of any predictor, I would say something like "The data did not provide support for the inclusion of any of the predictors (all inclusion BFs < x)"

Cheers,

E.J.

lemakru

Dear E.J.,

thank you so much for your answer, you helped me a lot!

I have one follow-up question regarding the moderation analysis in the way JASP is suggesting.

In the moderation guide, the JASP Team wrote the following: "For the present example, comparing the model without the interaction effect (M0) to the model with the interaction effect (M1) using the default JZS prior, we achieve a very large Bayes Factor (BF10 = 2.618⋅10+9), indicating that M1 is much more likely than M0."

As you can see in the following table, the value stated for BF10 above is actually the value of BFM. Was there a mix-up or can I (even though I don't understand the reason behind it yet) use the BFM-value to report my BF10?

Thank you a lot!

EJ

What, we wrote that?! It is not correct, in the sense that it mistakes the BF for a posterior odds (in other words, if M1 is deeply implausible a priori it may not be the most likely model; but perhaps this was clear from context).

About your question: the BF_10 equals 1 because the model is compared to itself. The second entry shows that the BF_10 is almost 0, indicating overwhelming support for the model that includes the interaction. This evidence is sufficiently strong to drive the model probability up from .50 to almost 1, which is what you see in the P(M|data) column.

EJ