Welcome!

Sign in with your CogSci, Facebook, Google, or Twitter account.

Or register to create a new account.

We'll use your information only for signing in to this forum.

Supported by

What denominator does the Cohen's d use on JASP??

As I'm getting different results when calculating Cohen's d with SD Pooled as the denominator to the results JASP is giving me.
I just want to make sure I'm reporting it correctly in my write up.

Cheers,

Comments

  • EJEJ Posts: 368

    I'll ask Johnny what's up. I think the issue came up before. Do you have a concrete example?
    Cheers,
    E.J.

  • bivabiva Posts: 6

    Here's a couple of examples with the JASP d and the SD Pooled d (calculated using Lee Becker's website):
    M1 = 66.5 ± 72.8 M2 = 36.5 ± 30.7 JASP d = 0.84 SD Pooled d = 0.54

    M1 = 10.3 ± 14.8 M2 = 2.3 ± 4.1 JASP d = 0.54 SD Pooled d = 0.73

    As you can see, they're coming up with vastly different results, both increasing and decreasing the ES, some even changing thresholds (from moderate to large for example).

    Cheers

  • JohnnyBJohnnyB Posts: 12

    Hi Biva,
    I just ran some examples through both JASP and Lee Becker's calculator, and it seems that they provide the same output whenever the sample sizes in both groups are equal. If the sample sizes between groups differ, they indeed provide different results.
    JASP calculates Cohen's d with the following R-code:

    num <- (ns[1] - 1) * sds[1]^2 + (ns[2] - 1) * sds[2]^2
    sdPooled <- sqrt(num / (ns[1] + ns[2] - 2))
    d <- as.numeric((ms[1] - ms[2]) / sdPooled) 
    

    Whereas Lee Becker does the following:

    sdPooled <- sqrt((sds[1]^2 + sds[2]^2)/2)
    d <- as.numeric((ms[1] - ms[2]) / sdPooled) 
    

    So the latter does not weigh the standard deviations by sample size to determine the pooled standard deviation. My guess would be that Lee Becker's calculator rests on the assumption that the sample sizes are equal.
    See also here:
    https://en.wikipedia.org/wiki/Effect_size#Cohen.27s_d
    http://stackoverflow.com/questions/15436702/estimate-cohens-d-for-effect-size
    Andy Field - Discovering Statistics using SPSS

    Cheers,
    Johnny

  • bivabiva Posts: 6

    Hi, thanks for the reply Johnny.
    It makes sense everything you've said, but my sample sizes are equal. I double checked to see if I'd missed any participants from the data set, but they're all there.

    Would it be possible to send you a Jasp file with the data set via Dropbox or something?

  • bivabiva Posts: 6

    I've also just ran the following through a couple of other online calculators and they all agree with the 0.54 result:
    M1 = 66.5 ± 72.8 M2 = 36.5 ± 30.7 JASP d = 0.84 SD Pooled d = 0.54

    I obviously don't want to report incorrect results, and I'm getting concerned about the rest of my results as well at the minute.

  • EJEJ Posts: 368

    Hi Biva,

    Yes, please send us your data set. You can just Email it to EJ.Wagenmakers@gmail.com and I'll forward it to Johnny.

    E.J.

  • bivabiva Posts: 6

    Hi,
    that's great, I've sent them from C Kirk

    Cheers

  • JohnnyBJohnnyB Posts: 12

    Hi Biva,
    Thanks again for your help! I took a look at the JASP files you sent us. I was under the impression that you were conducting two-sample t-tests, as you reported the means and sd's for both groups. However, in the JASP files I saw that you are conducting paired samples t-tests.
    This has repercussions for the way Cohen's d is calculated. For two-sample t-tests, the formula's I described previously are used, but for paired sample tests, the following formula/code is used:

    d <- mean(c1 - c2) / sd(c1 - c2)
    

    where c1 and c2 refer to the scores of the first sample and second sample respectively, so now the denominator is the standard deviation of the difference scores (instead of the pooled standard deviation). One can also compute Cohen's d from the paired samples t-statistic:

    d <- t / sqrt(n)
    

    For instance, for your test on the variables WinSigStrLanded and LossSigStrLanded, the paired samples t-statistic equals 4.443, with n = 28. Now, 4.443/sqrt(28) = 0.84.
    See also:
    https://stats.stackexchange.com/questions/201629/cohens-d-for-dependent-sample-t-test
    http://mandeblog.blogspot.nl/2011/05/cohens-d-and-effect-size.html
    https://en.wikipedia.org/wiki/Effect_size#Cohen.27s_d (the first part of that section is about Cohen's d for two-sample t-tests, and the second part about Cohen's d for paired samples t-tests)
    Please let me know if you have any additional questions, I'd be happy to help!
    Cheers,
    Johnny

  • bivabiva Posts: 6

    Ah that explains a lot! That's really helpful, thankyou!

    Further proof - as if it was needed - that I have a lot more reading and learning to do as well!

    Thanks again!

  • JohnnyBJohnnyB Posts: 12

    Hi Biva,
    Happy to help, I'm glad that it got sorted out!
    Cheers,
    Johnny

Sign In or Register to comment.