BF from non-parametric z/t/p?

SDavid

Dear JASPers,

I am working in (structural) brain imaging and the consensus that parametric methods should not be used, therefore a lot of bootstrapping, permutation testing, robust testing, etc is going on. As far as I learned from you at the JASP workshop, right now there is no way to do non-parametric t-tests with the R package/in JASP, but I still would like to do some BF based hypothesis testing and here is the idea I would like to ask you to check:

There was a post in Alex Etz's blog about the maximum BF regarding a given p-value. Is it OK conceptually that I do the non-parametric statistical analysis with frequentists methods, meanwhile correcting for the multiple comparisons and false-positives, etc., which is again a kind of consensus in the brain imaging community, and then I could calculate the theoretically maximum BF for the resulted z/t/whatever score?

The blog post: https://alexanderetz.com/2016/06/19/understanding-bayes-how-to-cheat-to-get-the-maximum-bayes-factor-for-a-given-p-value/

Cheers,
Szabolcs
(Sa-bol-ch)

EJ

Hi Szabolcs,

We will soon include the Vovk-Sellke maximum ratio in JASP. I think this is a pretty general rule, but I am not sure whether it applies after you do multiple comparisons. This sort of depends on how you deal with multiple comparisons in a Bayesian framework. I would be more inclined to compute the maximum value from the original p-value, and then apply a multiplicity correction to that. But I am not 100% sure. Sorry about that; we are working to include more Bayesian nonparametric methods. I believe that George Karabatsos has a package for that...let me see...yes, this is the title: A Menu--Driven Software Package of {B}ayesian Nonparametric (and Parametric) Mixed Models for Regression Analysis and Density Estimation.

Cheers,
E.J.

SDavid

Hi E.J.,

Thanks for the reply and that reference!

TBH I can't see a consensus in the brain imaging community how to do multiple comp. precisely. BF based hyp. testing is not that prevalent (in my area of research), so for the first try I am not worrying about the multiple comp. or any other correction step.
Generally I will just calculate a t from a permutation test, then calculate the maximum BF.

Cheers,
Szabolcs