Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Positive BF inclusion for interaction but model with interaction not best model

Hi,

I am contemplating how to use/interpret the BFinclusion factor in light of the principle of marginality used in JASP. I have a 2x2 between subject design where the BFinclusions for the respective main effects and the interaction are Main1: 0.225, Main2: 52.809 and Interaction: 3.042. Thus, while there is evidence for including the interaction, the model with the strongest support is the one only including main effect 2(BFm: 4.3, BF10: 53), while the model including both main effects and the interaction effect (BFm: 2.2, BF10:37) is clearly being "dragged down" by Main effect1.

It seems resonable to me to conclude that there is an effect of Main2 and an interaction. Although such a model is not tested in itself. How would you approach this situation?


Best,

August

Comments

  • Your conclusions seem correct. One option would be to report these results. Another option might be to fit and test your specified model (one main effect and the interaction) in R:

    library(BayesFactor)
    
    data(puzzles) # example of 2*2 data
    
    
    # This is what JASP does
    Models <- anovaBF(RT ~ shape*color, data = puzzles)
    
    
    # specific model
    new_model <- lmBF(RT ~ shape + shape:color, data = puzzles)
    
    
    all_models <- c(Models,new_model)
    all_models
    #> Bayes factor analysis
    #> --------------
    #> [1] shape                       : 0.6114754 ±0.01%
    #> [2] color                       : 0.6114754 ±0.01%
    #> [3] shape + color               : 0.3843659 ±2.11%
    #> [4] shape + color + shape:color : 0.1448699 ±1.13%
    #> [5] shape + shape:color         : 0.2291015 ±0.87%
    #> 
    #> Against denominator:
    #>   Intercept only 
    #> ---
    #> Bayes factor type: BFlinearModel, JZS
    
    
    # also get inclusion BFs:
    library(bayestestR)
    bayesfactor_inclusion(all_models)
    #>             Pr(prior) Pr(posterior) Inclusion BF
    #> shape            0.67          0.46         0.43
    #> color            0.50          0.38         0.62
    #> shape:color      0.33          0.13         0.29
    #> ---
    #> Inclusion BFs compared among all models.
    
    


  • Yes. Depends how principled you are about the principle of marginality, I guess :-)

    E.J.

  • edited May 31

    Thanks for the quick replies!

    By including Main1 in the null-model I (kind of) test the models, although now not the full model with Main1, Main2 and the interaction, and against a somewhat different null model. Is such an approach absurd, or could it be feasible? Maybe best just to move to a r-analysis as suggested though..


    /August

  • If you are interested in the interaction, you can also compare the 2-main effect model against the full model. With few models you don't always need the inclusion BF.

    Cheers,

    E.J.

Sign In or Register to comment.