T-Statistic vs. Cohen's d Bayes Factor Outputs
Good evening! I have a question regarding the Bayesian Summary Statistics module and some of the mechanics under the hood that I may be misunderstanding.
I am in the process of running a study that utilizes a lot of conversions of t-statistics into BF10. Well within the process, I considered switching to converting Cohen's d outputs into Bayes Factors instead as it may better answer my research questions. I converted a sample of 50 or so t-statistic outputs into Cohen's d figures, and then compared the Summary Statistics outputs from the Independent Samples T-Test submodule using the t-values versus the General Bayesian Test module using non-central d outputs (both with uninformed Cauchy priors, set at -0.707, 0.707 in the t-value submodule, and as X0 = 1, theta = 0, -Inf, Inf Truncation for General Bayesian Test).
What I found is that the outputs for the Cohen's d BF10 are magnitudes higher than the Independent Samples T-Test Bayes Factor outputs. To provide an example: with t = 4.43 and two groups of 22 participants each (and thus a Cohen's d of 1.335695), the Independent Samples T-Test BF10 output is 311.614...but the Cohen's d BF10 output is 369000000, which is a magnitude of ~1,184,157 times higher. I understand that Bayes Factors are, of course, not effect sizes, but I did not anticipate such a discrepancy in BF10 outputs after a simple standardization of the t-value.
Is there something that I am not understanding regarding the analysis of Cohen's d into a BF10 output? I would appreciate any insight you could provide!
-Dan
Comments
Hi Dan,
I am not 100% sure what you did, but the BF is based on relative predictive performance for observed data y. It so happens that the t-value and sample size is a sufficient summary of the data, so that no information is lost when you enter t and n instead of the raw observations. Now other statistics will be sufficient too, and if that is the case they should give the same result. But I see this refers to a comparison with the General Bayesian Test module, so I will attend the relevant team member to your query.
In principle the MLE for Cohen's delta and the standard error of Cohen's delta should allow a test that yields almost the same result.
EJ