Help for interpretation : significant p-value and anecdotical BF

kev_bague

Hello,

I'm conducting frequentist as well as Bayesian analyses. However I have difficulties to interpret data.

Indeed, results showed significant p-value, but anecdotal BF10 (< 3). Likewise, there are non-significant p-value but anecdotal B01 (< 3).

I'm reading article written by Wetzel et al. (2011) in link with this topic.

In both case I don't conclude about acception of H0 or H1 ?

And is there a link with statistical power ? Indeed I have small sample, can we imagine that more participants would conduct to increase the evidence for H1 or H0 of BFs ?

Have a nice day

Cheers,

KB

EJ

Hi KB,

Yes, more samples will eventually result in overwhelming support for H0 or for H1. If you have a p-value that is just significant it will almost always be the case that the BF is not compelling. P-values only consider the surprise under H0, and ignore the surprise under H1. For a series of examples see the posts on BayesianSpectacles.org. See for instance the series of posts "redefine statistical significance" (you can search on the site)

Cheers,

E.J.

kev_bague

@EJ thank you for your answer, and for this ressource very interesting which helps me.

For instance, I'm conducting a linear regression. Frequentist approach shows a non-significant p-value for the model including predictor, that it does not reject H0 (according to frequentist approach). Bayesian analysis shows an anecdotical evidence BF < 3 for null model against model including predictors (in some cases I have an anecdotical evidence for model including predictor). It is very complexe to suggest a conclusion to this results... I can't conclude it this case to the acception of H0 ?

Thank your for your time and your precious response.

Cheers,

KB

EJ

Hi KB,

Well, I think you would just conclude that there is weak evidence for the exclusion of the predictor. It is important, when the data do not support a strong conclusion, to resist the temptation to draw a strong conclusion anyway. You could go one step further and study the size of the effect given it exists. Maybe its size is still highly unclear, or perhaps it is likely to be small?

Cheers,

E.J.