RM ANOVA 95% CI

Comments

• EJ

  June 2020

  I'll contact the team member (Johnny) who should know.

  E.J.
• JohnnyB

  June 2020

  Hi Martin,

  Do you have a JASP file I could take a look at? I don't think anything has changed there, but it might be that a default option has changed, for instance using the standard error instead of the confidence intervals, or maybe whether to average across unused RM factors or not.

  Kind regards

  Johnny
• zgolot

  July 2020

  Hello Johnny,

  this JASP file contains a plot that changes when refreshing the graph.

  I don't think it's an issue of default options, because no matter what I try after the graph has refreshed, it never looks as the old graph.

  
  

  Best regards,

  Martin

  Experiment 1.jasp

  
  
• zgolot

  July 2020

  Hello,

  I've investigated a bit and it looks like JASP used to plot within-participant CIs and now simply plots "regular" CIs. I've computed both regular and within-participant CIs (following this method: https://motorbehaviour.wordpress.com/2014/10/31/confidence-intervals-in-within-participant-design-a-tutorial-on-the-cousineau-morey-method/) on the same dataset linked to the JASP file above and the "before-update" sizes of the CIs seem to match the within-participants CIs (see attached file, hope it's clear enough). My knowledge in statistics is sadly not deep enough to know which CI it is appropriate to plot and what the intended behavior should be.

  Best regards,

  Martin Constant

  95% CI.xlsx

  
  
• JohnnyB

  July 2020

  Hi Martin,

  I just looked into this, and indeed it seems that the within-participant CI calculation was not used anymore! I just fixed it, so it will be in the next version of JASP. Thank you very much for your keen eye!

  Kind regards,

  Johnny
• christof

  February 2022 edited February 2022

  Hi there,

  Does anyone know which method JASP uses to calculate the CIs in RM ANOVA plots?

  From the link that zgolot posted above it appears that there are different methods that could be used. A reviewer recently criticised the CIs in one of my JASP plots. I think the reviewer just used mean, standard deviation and sample size to check the CI, which is not the best method in this case I think (please excuse my lack of deep stats knowledge).

  Thanks,

  Christof
• JohnnyB

  February 2022

  Hi @christof ,

  We apply a correction to these confidence intervals, based on Loftus & Masson (1994) and Morey (2008). This is what we write in the help file:

  By selecting this option, error bars will be displayed in the plot. The error bars can either represent confidence intervals or standard errors. In order to get accurate confidence intervals and standard errors, the data are normalized by subtracting the appropriate participantʹs mean performance from each observation, and then adding the grand mean score to every observation. The variances of the resulting normalized values in each condition, and thus the size of the bars, no longer depend on the participant effects and are therefore a more accurate representation of the experimental manipulation. See Morey (2008) for a thorough discussion of this procedure.

  • Morey, R. D. (2008) Confidence Intervals from Normalized Data: A correction to Cousineau (2005). Tutorial in Quantatitative Methods for Psychology, 4(2), 61-64.
  • Loftus, G. R., & Masson, M. E. J. (1994). Using confidence intervals in within-subject designs. Psychonomic Bulletin and Review, 1, 476–490

  So if you would use the normal, uncorrected version, the within-participant variation is conflating the effect of the manipulation (either over- or underestimated). You could also point the reviewer to the references above.

  Kind regards

  Johnny
• christof

  February 2022

  Thanks, @JohnnyB !