# What denominator does the Cohen's d use on JASP??

biva
Posts:

**6**As I'm getting different results when calculating Cohen's d with SD Pooled as the denominator to the results JASP is giving me.

I just want to make sure I'm reporting it correctly in my write up.

Cheers,

## Comments

444I'll ask Johnny what's up. I think the issue came up before. Do you have a concrete example?

Cheers,

E.J.

6Here's a couple of examples with the JASP d and the SD Pooled d (calculated using Lee Becker's website):

M1 = 66.5 ± 72.8 M2 = 36.5 ± 30.7 JASP d = 0.84 SD Pooled d = 0.54

M1 = 10.3 ± 14.8 M2 = 2.3 ± 4.1 JASP d = 0.54 SD Pooled d = 0.73

As you can see, they're coming up with vastly different results, both increasing and decreasing the ES, some even changing thresholds (from moderate to large for example).

Cheers

14Hi Biva,

I just ran some examples through both JASP and Lee Becker's calculator, and it seems that they provide the same output whenever the sample sizes in both groups are equal. If the sample sizes between groups differ, they indeed provide different results.

JASP calculates Cohen's d with the following R-code:

Whereas Lee Becker does the following:

So the latter does not weigh the standard deviations by sample size to determine the pooled standard deviation. My guess would be that Lee Becker's calculator rests on the assumption that the sample sizes are equal.

See also here:

https://en.wikipedia.org/wiki/Effect_size#Cohen.27s_d

http://stackoverflow.com/questions/15436702/estimate-cohens-d-for-effect-size

Andy Field - Discovering Statistics using SPSS

Cheers,

Johnny

6Hi, thanks for the reply Johnny.

It makes sense everything you've said, but my sample sizes are equal. I double checked to see if I'd missed any participants from the data set, but they're all there.

Would it be possible to send you a Jasp file with the data set via Dropbox or something?

6I've also just ran the following through a couple of other online calculators and they all agree with the 0.54 result:

M1 = 66.5 ± 72.8 M2 = 36.5 ± 30.7 JASP d = 0.84 SD Pooled d = 0.54

I obviously don't want to report incorrect results, and I'm getting concerned about the rest of my results as well at the minute.

444Hi Biva,

Yes, please send us your data set. You can just Email it to EJ.Wagenmakers@gmail.com and I'll forward it to Johnny.

E.J.

6Hi,

that's great, I've sent them from C Kirk

Cheers

14Hi Biva,

Thanks again for your help! I took a look at the JASP files you sent us. I was under the impression that you were conducting two-sample t-tests, as you reported the means and sd's for both groups. However, in the JASP files I saw that you are conducting paired samples t-tests.

This has repercussions for the way Cohen's d is calculated. For two-sample t-tests, the formula's I described previously are used, but for paired sample tests, the following formula/code is used:

where c1 and c2 refer to the scores of the first sample and second sample respectively, so now the denominator is the standard deviation of the difference scores (instead of the pooled standard deviation). One can also compute Cohen's d from the paired samples t-statistic:

For instance, for your test on the variables WinSigStrLanded and LossSigStrLanded, the paired samples t-statistic equals 4.443, with n = 28. Now, 4.443/sqrt(28) = 0.84.

See also:

https://stats.stackexchange.com/questions/201629/cohens-d-for-dependent-sample-t-test

http://mandeblog.blogspot.nl/2011/05/cohens-d-and-effect-size.html

https://en.wikipedia.org/wiki/Effect_size#Cohen.27s_d (the first part of that section is about Cohen's d for two-sample t-tests, and the second part about Cohen's d for paired samples t-tests)

Please let me know if you have any additional questions, I'd be happy to help!

Cheers,

Johnny

6Ah that explains a lot! That's really helpful, thankyou!

Further proof - as if it was needed - that I have a lot more reading and learning to do as well!

Thanks again!

14Hi Biva,

Happy to help, I'm glad that it got sorted out!

Cheers,

Johnny

1Hi,

Sorry for opening again this discussion, but I wonder why JASP team decided to compute Cohen's d for paired samples the way JohnnyB described, with the standard deviation of the difference scores (SD_diff) as denominator. Cummings (2012) argued against that and Dunlap, Cortina, Vaslow and Burke (1996; Psychological Methods) showed that computing d from the t statistic largely overestimates the effect size because it does not account for the correlation between samples.

I just tried with the data attached (separator is ";").

- In JASP, paired samples t-test shows Cohens d = 0.780, t = 2.466, and r = 0.962.

- Following the formula in Table 1 of Dunlap et al. (1996), d = t[2(1 - r)/n] (note the correlation in the formula), we get d = 2.466* sqrt(2* (1-0.962)/10) = 0.215

- If Cohen's d is computed following Cumming (2012, formula 11.10) and using SDPooled as denominator as JohnnyB posted above (either formula, because n is always the same for paired samples):

sdPooled <- sqrt((sds[1]^2 + sds[2]^2)/2)

you get d = (mean1 - mean2) / sdPooled = 5.2 / 21.90 = 0.237.

If you correct d = 0.237 for bias and get Hedge's g, then you get d_unbiased = 0.217, very similar to the outcome with the formula of Dunlap et al. (I may have lost some decimals here and there). In any case, d is more similar using sdPooled and the Dunlap et al. (1996) formula than with JASP and SD_diff.

In general, it seems to me that sdPooled may be a better denominator than SD_diff for paired samples, but I wonder what's the opinion of the JASP team in this matter.

Cheers,

Karlos