Cohen's d effect size not showing in 2x2 ANOVA post hoc
Hi, thanks for Jasp - it's a really great tool!
I have noticed that when running a 2x2 RM ANOVA, the effect size does not show up in the interaction post hoc tests, but it does show up in the main effect post hocs (same question was asked here: https://forum.cogsci.nl/discussion/6417/cohens-d-does-not-show-in-the-interaction-post-hoc-comparisons).
I ran the corresponding paired t-tests to obtain this, but the t statistic is different here, so it makes me think there is something I'm missing. What is the recommended approach to get the correct cohen's d value for the interaction post hocs?
Thanks
Comments
Looking at this further, I think the output (at least in the RM ANOVA case) seems incorrect somehow, or I'm not understanding something.
The Jasp values disagree with the same comparisons in both SPSS and in R (using the
pairwise_t_test()function ofrstatix). Not by much, but for example the p-value (Bonferroni) from Jasp tells me .0858, whereas the Bonferroni corrected p-value from R is saying it is .116. R's uncorrected p-value is .019, which is the same as what SPSS tells me (if I add the syntax line: ` /EMMEANS=TABLES(attention*item) COMPARE(attention) ADJ(BONFERRONI)` to my ANOVA). So R and SPSS seem the same, only SPSS is showing me an uncorrected value. Whereas Jasp is giving me .0858 from somewhere 🤔Edit: just tried the same thing in Jamovi and Jasp is still the outlier! I haven't tested on other data but it seems like it could be a bug
I've passed this on, we'll be in touch!
E.J.
Hi @num3 ,
The effect sizes (inc their confidence intervals and multiplicity correction) will be included for interactions in the next JASP release!
As for the difference in p-values - you could try ticking the box "use multivariate model for follow-up tests" in the "Model" tab. If I remember correctly, the multivariate model handles sphericity violations better. I thought we left this option unchecked by default to mimic SPSS results, but maybe something changed in their settings. Could you check this and get back to me?
Kind regards
Johnny
Great to hear they will be included! And I can confirm that after checking "Use multivariate model for follow-up tests" all of the values agree, so that seems to have been the issue.
Thanks!