Bayesian Optional Stopping while reporting P Values

Whirly123 · May 2023

Suppose I was gathering data based on on a stopping rule like stop when either the BF is above 10 or bellow 1/10.

Is there any way to also report p-values on that data. I know you can do sequential analysis using with p values but that is based on different stopping rules. Does the p value make sense given the data one has eventually gathered? Are there corrections one can do in order to report p values along-side BFs when doing Bayesian optional stopping ?

EJ · May 2023

Hi Whirly123,

This is an interesting question. Note that a BF (for H0 vs H1) is related to a p-value approximately as 3*p*sqrt(n), see https://psyarxiv.com/egydq

So in a way a Bayesian does optional stop on a p-value, but with a sample size correction. I am however still not sure whether the data may be analyzed with a p-value and retain their usual frequentist properties. In general, the BF bounds will respect the Universal Bound, but this bound need not be the same as the alpha-level that a frequentist would set. So I am inclined to say that formally, this cannot be done. This may be studied in simulation work, but somebody must have already looked at it.

Cheers,

E.J.

Howdy, Stranger!

Categories

Bayesian Optional Stopping while reporting P Values

Comments

Howdy, Stranger!

Quick Links

Categories

Bayesian Optional Stopping while reporting P Values

Comments