rmAnova with covariate

Comments

• FeB

  July 2023

  Hi,

  I would like to expand the question and provide a little more detail.

  We have two factors with two levels each (myF1: A/B, myF2 = C/D). These are just 4 conditions (AC, AD, BC, BD) each participant responded to (repeated measures). We have a group factor (group1, group2) and a covariate (let’s say a continuous factor like age but zscored).

  In general, we’re interested in the effect the covariate can have on the repeated measures (we do not want to cancel it out but rather find whether e.g. the interaction of Factors 1 & 2 interacts with the covariate).

   

  Question 1: From what we were able to gather from the Internet, you can always use repeated measures in SPSS or JASP and enter a covariate -- which is either a nuisance regressor or regressor of interest à as the model is agnostic to this difference, it does not matter what it is. Is this correct?

   

  Question 2: If we add a covariate in JASP in the rmANOVA, are all effects in the output table based on mean adjusted values (values – the effect of the covariate)?

   

  Question3: If we run post hoc tests, would these show the mean adjusted values?

   

  Question 4: Maybe this is partially answered by Question 2. What would be the difference between an interaction of “myF1” vs. “myF1 * covariate” or, for instance, “myF1 * myF2” vs. “myF1 * myF2 * covariate”? Are in case 1 the main effects and in case two the double interactions always mean adjusted (independent of the interaction with the covariate)?

   

  Question5: And if we take “myF1 * covariate” from Question 4 as an example. Does this interaction only imply a difference in the mean adjusted values of the levels A and B for myF1. Or does it additionally OR alternatively imply that each level correlates differently with the covariate. I hope this question is not too confusing.

   

  Question 6: If we use the long format and analyse the data with the ANCOVA option, we do not get the same results and we do not get interactions with the covariate. So what is the difference between the repeated measure ANOVA with covariate and the ANCOVA. Shouldn’t both be called ANCOVA because we add the covariate and thus, yield similar results? And given the difference in results, we guess that you should never use the ANCOVA option when analyzing repeated measures data?!

   

  Find an image showing the output for the rmANOVA (wide format) and the ANCOVA (same data long format) below.

   

  

  
  

  Thanks for your help! Best, Felix
• EJ

  August 2023

  I'll ask our expert! (Sorry for the tardy response)
• JohnnyB

  August 2023

  Hi @kay_P & @FeB ,

  I'll reply to each question of FeB below, because I think they also cover kay's question:

  Question 1: From what we were able to gather from the Internet, you can always use repeated measures in SPSS or JASP and enter a covariate -- which is either a nuisance regressor or regressor of interest à as the model is agnostic to this difference, it does not matter what it is. Is this correct?

  You can indeed always add a continuous covariate to a repeated measures ANOVA, either out of theoretical or methodological interest. If you want to assess any interference of age, for instance, you can add it to the model to infer if there is an effect of age on the dependent variable, or if any interaction effects are present (which makes interpreting the main effects trickier). There's a strong distinction between exploratory and confirmatory here though (see for instance https://link.springer.com/article/10.3758/s13423-015-0913-5).  Note that in the RM ANOVA, we always add all interactions between the RM factors and the fixed factors/covariates because of this.

  
  

  Question 2: If we add a covariate in JASP in the rmANOVA, are all effects in the output table based on mean adjusted values (values – the effect of the covariate)?

  Yes, the model that is specified is used for the main analysis and its follow-up analyses.

   

  Question3: If we run post hoc tests, would these show the mean adjusted values?

   In the frequentist ANOVA these are based on the full model (see my blogpost here: https://jasp-stats.org/2020/04/14/the-wonderful-world-of-marginal-means/), so if a covariate is added, the marginal means (and therefore posthoc and contrasts) are estimated such to keep covariates in balance.

  Question 4: Maybe this is partially answered by Question 2. What would be the difference between an interaction of “myF1” vs. “myF1 * covariate” or, for instance, “myF1 * myF2” vs. “myF1 * myF2 * covariate”? Are in case 1 the main effects and in case two the double interactions always mean adjusted (independent of the interaction with the covariate)?

  The difference here is the same as with any other 2-way interaction vs. 3-way interaction: in the former, it means there is a different effect of a variable, based on another variable, while in the latter that different effect based on another variable, is based on yet another variable. I don't like these very much since they are difficult to interpret (perhaps 3-way is still somewhat intuitive, but from 4-way interactions it gets pretty insane).

   

  Question5: And if we take “myF1 * covariate” from Question 4 as an example. Does this interaction only imply a difference in the mean adjusted values of the levels A and B for myF1. Or does it additionally OR alternatively imply that each level correlates differently with the covariate. I hope this question is not too confusing.

  The interaction can be viewed from two angles I guess:

  If you find evidence for that interaction effect, it means that there is a correlation between your DV and covariate, but differently for A and B. Alternatively, the extent of the group difference of your DV for A and B depends on the value of the covariate.

  To get a better grip on these differences I would suggest a posthoc test.

   

  Question 6: If we use the long format and analyse the data with the ANCOVA option, we do not get the same results and we do not get interactions with the covariate. So what is the difference between the repeated measure ANOVA with covariate and the ANCOVA. Shouldn’t both be called ANCOVA because we add the covariate and thus, yield similar results? And given the difference in results, we guess that you should never use the ANCOVA option when analyzing repeated measures data?!

  I think the analyses differ from each other, in the same way a paired t-test and and independent samples t-test differ from each other. I am a bit hazy on the specifics, but RM factors get treated differently from between factors due to the observations being matched (one of the benefits here being that baseline differences in the DV are accounted for).

  Hopefully this clarifies your questions - please let me know if any further babbling is needed!

  Johnny
• FeB

  October 2023

  Dear Johnny,

  Sorry for the late reply! And many, many thanks for this very detailed reply. That helped a lot!

  Felix