95% CI for Cohen's d in post-hoc tests
Is it possible to report a pBonf = 0.03 with a 95% CI that contains a zero value?
I attach a screenshot with the values reported by JASP, please see second row of the attached table.
Thanks in advance.
Comments
Hi @vicente_inefo ,
My guess here is that the p-values are for the t/mean difference value (you can see that the CI for the mean difference does not include 0), which use a slightly different standard error for their standardization and CI computation, which can result in these types of fringe cases. I would think that if the result is this close, there is not very substantial evidence for a difference between Match and TriveBk.
Kind regards,
Johnny
Thanks for the reply @JohnnyB ,
Then I think is better not to report the Cohen's d in the paper we are writing because it could be confusing, isn't it?
Hi @vicente_inefo ,
I think this is a problem that stems from binary reasoning about statistical results - if you hold the 0.05 threshold as very holy, and a p-value below that as plain evidence for a difference, then yes, this Cohen d CI is a bit confusing because it illustrates that there is just not very much evidence in your data for the difference. I would report the full results, and curb the conclusion a bit (i.e., "weak evidence" or "tentative") about the difference, but of course this is very easy for me to say, being at the sideline and all ;-)
Kind regards,
Johnny
I understand your point but the reviewers, as a general rule, tend to "suggest" the use of this binary reasoning about statistical results. It's a pitty.