RM ANOVA 95% CI

zgolot

Hello,

I noticed in the last release of JASP (0.12.2) that 95% CI in the RM ANOVA plots have changed (i.e. they're usually wider than they used to be). Is there a reason for that, was there (or is there now) a bug?

Also, I am not well versed in statistics but I was wondering how I could get the values for these CI as JASP does not output them in a table form.

Best regards,

Martin Constant.

EJ

I'll contact the team member (Johnny) who should know.

E.J.

JohnnyB

Hi Martin,

Do you have a JASP file I could take a look at? I don't think anything has changed there, but it might be that a default option has changed, for instance using the standard error instead of the confidence intervals, or maybe whether to average across unused RM factors or not.

Kind regards

Johnny

zgolot

Hello Johnny,

this JASP file contains a plot that changes when refreshing the graph.

I don't think it's an issue of default options, because no matter what I try after the graph has refreshed, it never looks as the old graph.

Best regards,

Martin

Experiment 1.jasp

zgolot

Hello,

I've investigated a bit and it looks like JASP used to plot within-participant CIs and now simply plots "regular" CIs. I've computed both regular and within-participant CIs (following this method: https://motorbehaviour.wordpress.com/2014/10/31/confidence-intervals-in-within-participant-design-a-tutorial-on-the-cousineau-morey-method/) on the same dataset linked to the JASP file above and the "before-update" sizes of the CIs seem to match the within-participants CIs (see attached file, hope it's clear enough). My knowledge in statistics is sadly not deep enough to know which CI it is appropriate to plot and what the intended behavior should be.

Best regards,

Martin Constant

95% CI.xlsx

JohnnyB

Hi Martin,

I just looked into this, and indeed it seems that the within-participant CI calculation was not used anymore! I just fixed it, so it will be in the next version of JASP. Thank you very much for your keen eye!

Kind regards,

Johnny

christof

Hi there,

Does anyone know which method JASP uses to calculate the CIs in RM ANOVA plots?

From the link that zgolot posted above it appears that there are different methods that could be used. A reviewer recently criticised the CIs in one of my JASP plots. I think the reviewer just used mean, standard deviation and sample size to check the CI, which is not the best method in this case I think (please excuse my lack of deep stats knowledge).

Thanks,

Christof

JohnnyB

Hi @christof ,

We apply a correction to these confidence intervals, based on Loftus & Masson (1994) and Morey (2008). This is what we write in the help file:

By selecting this option, error bars will be displayed in the plot. The error bars can either represent confidence intervals or standard errors. In order to get accurate confidence intervals and standard errors, the data are normalized by subtracting the appropriate participantʹs mean performance from each observation, and then adding the grand mean score to every observation. The variances of the resulting normalized values in each condition, and thus the size of the bars, no longer depend on the participant effects and are therefore a more accurate representation of the experimental manipulation. See Morey (2008) for a thorough discussion of this procedure.

• Morey, R. D. (2008) Confidence Intervals from Normalized Data: A correction to Cousineau (2005). Tutorial in Quantatitative Methods for Psychology, 4(2), 61-64.
• Loftus, G. R., & Masson, M. E. J. (1994). Using confidence intervals in within-subject designs. Psychonomic Bulletin and Review, 1, 476–490

So if you would use the normal, uncorrected version, the within-participant variation is conflating the effect of the manipulation (either over- or underestimated). You could also point the reviewer to the references above.

Kind regards

Johnny

christof

Thanks, @JohnnyB !