Bayesian one sample t-test versus Wilcoxon signed-rank: completely different result in JASP?
Hello,
I want to compare my data sample to an established value and I've been using a Bayesian one sample t-test in JASP 0.13.1 to do so. However, one of my data samples violates the normality assumption so I switched to the Wilcoxon signed-rank within the same feature in JASP. I noticed that the results are completely different! E.g., one sample has a mean of 63.9 (SE 1.6) and the t-test gives me a BF+0 of 2.9e+6 when comparing this with 50, strongly indicating that my sample deviates from 50. However, switching to the Wilcoxon with the same sample results in a BF+0 of 0.270 indicating no difference from 50 which seems very strange to me.
Could this be a bug? Doing the non Bayesian Wilcoxon also gives me a highly significant difference from 50.
Best,
Max
Comments
Hi Max,
Could you rerun this with the last version of JASP (0.14)? I changed the underlying sampling algorithm, since it had a bug that would sometimes lead to very conservative results, just like you are describing. If you still have this issue please let me know!
Cheers
Johnny
Hey,
Thanks for your fast reply! Unfortunately, I still have the same issue after updating.
Best,
Max
Hi Max,
I just looked, and the BF's I get for the one-sample Wilcoxon tests seem to be in line with the frequentist and parametric results. Would you comfortable sharing your data set with me? That way I can take a closer look. You can also send it by email to j.b.vandoorn <at> uva.nl
Kind regards,
Johnny
Sure,
my minimal example is the attached data set. I have done a Classical as well as Bayesian one sample t-test and Wilcoxon testing a significant larger value than 50. All but the Bayesian Wilcoxon are significant.
I think it is because you do the one-sided test, and the sidedness changes from regular t-test to Wilcoxon. You can see this when you do the two-sided test first; for the Wilcoxon, most mass is on the negative values. So you should compute BF-0 for the Wilcoxon. Johnny, this strikes me as suboptimal?
E.J.
Yes, something is wrong, I fixed it now. The test value was not considered correctly for the one sample wilcoxon...
Okay, will this then be in another release or should I take the BF-0 like EJ is suggesting?
Thanks for your help!
The next release is a bug-fix release, due out in a few weeks. But if you want to proceed right now you can just take BF-0 for the Wilcoxon: that will give you the right result.
Hi Max,
I fixed it and ran the correct code, see below!
I also have other comparisons for which I would like to run the Wilcoxon and the BF-0 still gives me quite different results. Is there a way to get the correctly working version for my PC? My email is 10maxgold (at) gmail.com
Reporting BF-0 solved the earlier issue. The new release will fix the labeling error, but you now have a way to get to the right number. If I understand correctly, you now report a new issue, which is that there are still "quite different results". Can you give an example?
@EJ we resolved the issue over email, the issue was not the flipping, but that the test value was not taken into account (and just set to 0). The nightlies should already contain the fix, and Joris informed me that the windows nightly will be working from tomorrow on.
Ahhh my bad, good to know it's fixed