Interpretation of Data (specifically interaction)

KevinvSchie

Hi all,

I have a question about the interpretation of an interaction. I have attached the output as a picture. Here I would argue that a one factor model of Time (pre vs. post intervention) is best supported by the data. Adding Group as an additional factor makes the BF smaller (by approx. a factor 5). Here comes the tricky part. How should I interpret the two factor model + interaction compared to the one factor model of Time, and specifically: does the data support an interaction? It is slightly better (by a factor 1.25). I would argue that it doesn't add a lot compared to Time Only. My supervisor argues that the two factor+interaction model isn't worse compared to the Time Only model (actually slightly better). So, how should I interpret this correctly. Is there evidence for an interaction or isn't there? Thanks in advance for any advice on this!

EJ

The model with the interaction is the best. To see by how much, it is easiest to select "compare to best model".

Cheers,
E.J.

KevinvSchie

Hi E.J.,

The model with the interaction is 1.25 better than a model with Time only. It's only better by a bit, but would you then conclude that there is support for an interaction? Or would you be more careful in stating this?

EJ

That level of evidence is almost completely nondiagnostic. If you have strong reasons to include the two main effects then you can focus on the two main effect model compared to the full model that also includes in the interaction (this is the same as defining the two main variables "as nuisance"). If the approach is more exploratory than you can explain the complete table.

KevinvSchie

I specifically expected an interaction between Group and Time, so to assess whether this is supported by the data I always need to compare it to the two main effects model (i.e., adding these as nuisance factors) and cannot compare it to a main effect model of Time. Is this correct?

KevinvSchie

What I was wondering; is it accepted to add Group and Time as nuisance factors and then state that there is no support for an interaction, and then follow that up with describing the complete table?

EJ

If you add group & time as nuisance, then you are comparing the full model to the two-main effects model. This yields BF=6.25 in favor of adding the interaction. So that's a little better than "no support"

KevinvSchie

Hm, I thought I had this figured out, but I started reading about the 'Baws factor' (which it seems is an arbitrary term), but is implemented in the latest version of JASP under the 'Output' and then 'Effects' boxes.

One of my reviewers argues that for the interaction I should actually use this part of JASP. I am not sure if I need to do this, if I want to show evidence for the interaction. And if so, should I then use 'across all models' or 'across matched models'?

EJ

I'd report report the entire model comparison table. Yes you can test the two main effects model against the model that also includes the interaction. It seems to me that this is the most intuitive analysis. Maybe the table will look more compelling if you select "compare to best model" and then "BF_01": you will see the best model on top, and then the BFs will indicate how much more support the best model receives compared to each of the others.

Cheers,
E.J.