Effectsize comparison with Frequentist and Bayesian Mann Whitney test

Comments

• EJ

  July 15

  Hi Mirjam,

  Our Bayesian Mann-Whitney expert is enjoying a holiday, but I have attended him to your post!

  Cheers,

  E.J.
• Mirjam

  July 16

  Thanks a lot!!

  Mirjam
• andersony3k

  July 28

  Best to wait for the expert to return. However, in playing around with some numbers, it's clear to me that the median 'effect size' in the posterior plot is not the median rank-biserial correlation. This is because notice that the median 'effect size' sometimes goes well above 1.0, whereas the rank-biserial correlation is constrained to vary between -1 and 1.

  R
• JohnnyB

  September 2

  Hi @Mirjam ,

  As @andersony3k rightfully pointed out, there are different effect sizes at play here. The rank-biserial correlation is a correlation and is bound to be between -1 and 1, while the effect size for the Bayesian Wilcoxon is Cohen's d based on data imputation, so can also be greater than 1.

  The method is explained in our paper here, in case you want to read more.

  Kind regards,

  Johnny
• Mirjam

  September 5

  Hi Johnny,

  I hope you had a great holiday!

  Thanks so much for your answer and the paper! Thanks also to @andersony3k

  Kind regards,

  Mirjam
• andersony3k

  September 5

  @JohnnyB Thanks. From the paper, I'm seeing that the latent-normal assumption applies when you have ranked data that might well have been normal had they not been converted into ranks. However, I believe a major motivation for performing a Mann-Whitney test is when the analyst has good reasons not to accept the normality assumption that underlies a t t-test. In that common situation, latent normality assumption has largely been rejected already, prior to subjecting the decidedly non-normal data to ranking. So in this latter situation, JASP's Bayesian approach wouldn't be quite applicable. Is that correct?

  R
• EJ

  September 5

  Let me speak for Johnny, as I know he's busy teaching. The assumption (and yes, it is a parametric assumption) is that the latent distribution is normal. But my current thinking is that this is a relatively mild assumption, since it applied also to any monotonic transformation of the raw data. Suppose for instance your RTs are not normally distributed, but heavily skewed. The latent normal distribution could be on "drift rate" in a diffusion model; this translates to a non-normal distribution on RTs. Another way to view this is that you can take the non-normal raw data, apply any monotonic transformation to achieve (approximate) normality, and then assume that all you know are the ranks.

  EJ
• andersony3k

  September 6 edited September 6

  @EJ

  OK. But maybe the situations in which you need non-parametrics the most are when such a transformation is not possible because the two non-normal distributions have different shapes (as illustrated below).

  

  
  

  R
• EJ

  September 6

  Yes, but in these situation a rank-based analysis would not be a good test either...
• andersony3k

  September 6

  I think a rank-based analysis is fine in these situations as long as one doesn't treat the test as going beyond ranks in any way. The statistical inference concerns whether, in the population, the Group A values tend to rank differently (i.e., higher or lower ) than do the Group B values. (Nothing to do with means, medians, or anything besides ranks.)

  R