Effect size for Wilconxon signed-rank test

pedromoreira

Hi,

I am running a non-parametric paired samples analysis.

I've been reading about calculation of the effect size r for this analysis and most literature referes to the formula proposed by Rosenthal (1991). The formula is: r = Z/sqrt(N). In the case of JASP, the way the same coefficient r is computed seems to be quite different: W / ((n*(n+1))/2) * 2 - 1 (thank you Johnny van Doorn for the clarification).

If we assume N as the number of subjects (e.g., 10 subjects evaluated in two moments), the calculation of this effect size is (in my example) varies from r=.693 to r=.782, depending on whether the calculation is based on the Rosenthal formula or as it is implemented in JASP.

To add even more noise to this, Field (2009) and Pallant (2007) propose that when dealing with related samples, N should refer to the number of observations (i.e., 10x2 in my example) and thus this difference is even more pronounced (r=.490). I am reporting such analysis in a revision of a manuscript and I would like to have more certainty in how I should compute this ES, given the impact that this has for the magnitude of the effect.

Do you have any thought on the most accepted way of computing this?

Thank you very much for your attention

Best wishes

Pedro

JohnnyB

Hi Pedro,

To follow up on my tweet, here is an excerpt of an article we are currently working on (about the Bayesian version of the various nonparametric t-tests, found here https://arxiv.org/abs/1712.06941):

"An often used standardized effect size for $W$ is the matched-pairs rank-biserial correlation, denoted $\rho_{mrb}$, which is the correlation coefficient used as a within subjects measure of association between a nominal dichotomous variable and an ordinal variable (Cureton 1956, Kerby 2014}. The transformation is as follows:

 $\rho_{mrb} = 1 - \frac{4W}{n ( n+1)}$. # equivalent to formula I gave you

The matched-pairs rank-biserial correlation can also be expressed as the difference between the proportion of data pairs where $x_i > y_i$ versus $x_i < y_i$. "

Basically, the formula we use is just a way to standardize the W statistic to cover the range [-1, 1]. I really like the straightforward and intuitive interpretation that it provides in terms of difference in proportions of data pairs where $x_i > y_i$ versus $x_i < y_i$.

Could you tell me what Z you are talking about? Is the the normal approximation/transformation of the W statistic?

Kind regards,

Johnny

pedromoreira

Dear Johnny,

Thank you very much for your feedback.

The formula that I am referring to is computed as follows: https://www.statisticshowto.datasciencecentral.com/wilcoxon-signed-rank-test/. The lowest sum of ranks (i.e., the lowest value between the sum of positive ranks and the sum of negative ranks) is used together with n to compute the Z-statistic.

A typical approach is to obtain this value is to rely on the significance of the Wilcoxon test

(qnorm(Model$p.value/2) (as reported by Andy Field)

It is one of the outputs obtained with SPSS (and other commercial software) and is widely reported when researchers present the results of this test.

Best wishes,

Pedro