Evaluating Regression Models in JASP
Hello everyone,
I have a question regarding this passage from 'Bayesian Inferences for Psychology Part II' (p. 20):
'All models (except perhaps for Pitch) receive over- whelming evidence in comparison to the Null model. The model that outperforms the Null model the most is the two main effects model, Gender + Pitch. Adding the interaction makes the model less competitive. The evidence against including the interaction is roughly a factor of ten. This can be obtained as 8.192e+39 / 8.864e+38 ≈ 9.24. Thus, the data are 9.24 times more likely under the two main effects model than under the model that adds the interaction.'
My table from JASP looks like this (I hope the attached screen shot works). How do I use the above technique here? Do I always just choose the model with the larger BF10 and divide it by the smaller BF10 to find out which main effect makes the model less competitive?
My attempt at writing this up currently looks like that:
"Table 11 shows that the model which was most strongly supported was the one with the main effects of Calm + Friendliest (BF10=58.37). For this model, there was evidence against the inclusion of Calm (BF=9.11) but not against the inclusion of Friendliest (BF=1.80) which suggests Friendliest was the more important predictor of Winter Again. The Calm + Friendliest + Leadership model is also strongly supported (BF10=54.97) and there was no evidence against the inclusion of Leadership (BF=1.04). Additionally, there was strong support for the model combining all four aspects: Calm + Friendliest + Closest Friend + Leadership (BF10=16.06). So, There was moderate evidence against including Friendliest in this model."
Comments
The file with the screenshot I wanted to upload has repeatedly failed to upload...I hope this link works!
https://ibb.co/b4juVk
Hi Eniseg2,
The next version of JASP will have the option to put the best model on top as a comparison model, which will make all of this easier. With respect to your text, my comments inline:
"Table 11 shows that the model which was most strongly supported was the one with the main effects of Calm + Friendliest [EJ: FriendliestMostPupular, I assume ] (BF10=58.37) [EJ: this is against the model without any predictors]. For this model, there was evidence against the inclusion of Calm (BF=9.11) [EJ: You might want to clarify which models you are comparing here -- also, there can't be evidence against the inclusion of Calm if Calm + Friendliest is the best model] but not against the inclusion of Friendliest (BF=1.80) which suggests Friendliest was the more important predictor of Winter Again. The Calm + Friendliest + Leadership model is also strongly supported (BF10=54.97) [EJ: but again, this is compared to the null model without any predictors] and there was no evidence against the inclusion of Leadership (BF=1.04). Additionally, there was strong support for the model combining all four aspects: Calm + Friendliest + Closest Friend + Leadership (BF10=16.06). So, There was moderate evidence against including Friendliest in this model."
I think what you want to look at is the analysis of effects, the table that computes inclusion probabilities by averaging across all of these models. Your example is linear regression, I assume.
Cheers,
E.J.