Computation details - ttests and correlations
Hello,
I'm using Bayesian statistics for the first time on Independent Samples T-Tests and Kendall's Rank Correlations on JASP.
After I presented the results to my colleagues, one of them responded with the following comment : ''Before presenting any Bayesian analysis results, you need to present convergence diagnostics. In the methods you need to state how many MCMC simulations you ran, and in the results you need to show that the models converged.''
A recent paper (Kruschke, 2021) presented reporting guidelines and mentionned the importance of this information. They suggested reporting the Potential Scale Reduction Factor (PSRF or R-hat) and the ESS (number of steps).
I have looked everywhere I could think of for these values but have not found them in JASP options and outputs.
Thank you for your guidance,
Emma
Comments
Hi Emma,
For the regular t-test, no MCMC is needed. In fact we try to avoid MCMC whenever we can. Not all Bayesian inference uses MCMC! However, for Kendall's tau we do use a sampling-based methodology and you are right, we don't give Rhat, and I think we should. We do provide Rhat for our rank-based t-tests, and allow the possibility to set the number of samples. I'm not sure why we don't do that for Kendall's tau. Maybe there is a good reason, but I think it is simply an omission. We'll deal with this, thanks for pointing it out.
Cheers,
E.J.
Hi E.J.!
Thanks so much for your quick answer!
Just to confirm, at the moment, is there no way to obtain the Rhat for the Kendall's tau?
Thanks again!
Emma
Hi Emma,
Not at the moment in JASP, although perhaps you can with Johnny's R code -- I'll ask him.
E.J.
Thank you so much ! Let me know :)
Hi @emmacampbell,
The method that is currently in JASP is not MCMC-sampling based, so there are no convergence or sampling diagnostics to report. I am planning an overhaul of the analysis to update the (sampling-based) method, and then we will have these diagnostics. Please let me know if you have any further questions.
Kind regards
Johnny
Thank you for your answers. I very much appreciate you taking the time!