Different Cohen's d values for rmANOVA post hoc tests and paired sample t-tests

je77je

Hey :)

Could someone please explain to me, why i get different Cohen's d values for rmANOVA post hoc tests and paired sample t-tests? I used the same 3 groups for calculation.

Is it maybe related to a different denominator? Is JASP calculating Cohen's d_z in both cases?

Thanks for your help :)

EJ

This is a recurring topic on this forum :-)

I noticed a previous thread that may be relevant: https://forum.cogsci.nl/discussion/3013/what-denominator-does-the-cohens-d-use-on-jasp

Does this address the issue? If not I'll ask Johnny to look at this.

E.J.

je77je

Thank you for the quick responde :)

I already saw that thread but it was a bit too difficult for me to understand :/

I'm pretty new to this stuff, just doing my first statistical analysis. Maybe you (or Johnny) could break it down for me?

Sorry for the circumstances and thank aou for your help :)

je77je

Hey :)

Are there any updates on my request?

Sorry for bugging, but I have to finish my work in an few weeks and would really like to use correct information.

Either way, thank you for your help this far :)

JohnnyB

Hi @je77je ,

Sorry for the delayed response, this flew under my radar! The reason for the discrepancy you see, is that by default, the RM ANOVA post hoc tests use the pooled standard error (you can see that the SE is the same for all 3 comparisons). This corresponds to the RM ANOVA statistical model, which specifies that each of the RM levels belong to the same RM factor, and so the errors should follow the same distribution (and so have the same standard deviation). Doing individual t-tests ignores this bit, and so uses individual SE's for each comparison. We have the option to use the non-pooled SE in the RM post hoc test (see the checkbox in that analysis), which should lead to the same results as the paired t-tests. However, we advise the use the pooled SE, as this corresponds more closely to the RM ANOVA model itself.

Kind regards,

Johnny