Bayesian Optional Stopping while reporting P Values
Suppose I was gathering data based on on a stopping rule like stop when either the BF is above 10 or bellow 1/10.
Is there any way to also report p-values on that data. I know you can do sequential analysis using with p values but that is based on different stopping rules. Does the p value make sense given the data one has eventually gathered? Are there corrections one can do in order to report p values along-side BFs when doing Bayesian optional stopping ?
Comments
Hi Whirly123,
This is an interesting question. Note that a BF (for H0 vs H1) is related to a p-value approximately as 3*p*sqrt(n), see https://psyarxiv.com/egydq
So in a way a Bayesian does optional stop on a p-value, but with a sample size correction. I am however still not sure whether the data may be analyzed with a p-value and retain their usual frequentist properties. In general, the BF bounds will respect the Universal Bound, but this bound need not be the same as the alpha-level that a frequentist would set. So I am inclined to say that formally, this cannot be done. This may be studied in simulation work, but somebody must have already looked at it.
Cheers,
E.J.