Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Question about BF and sample size.

Hi, I have a question regarding increasing participants numbers and the associated BF that I was hoping someone could help me with.

We have two studies, one with approximately 200 participants and one with 400 participants. When running a Bayesian contingency table on the data, we find that the BF for the first study is larger than for the second study (BF ~ 4.2 reduces to BF ~ 3.2). As the two designs are very similar, we combined the data set, and the Bayes Factor reduces even further (BF ~ 1.6).

Is this simply because as we increase the sample number, we're getting a better estimate true distrubtion, and our BF is changing to reflect that? Or is there something else that might be causing this?

Thanks for your help.

-J

Comments

  • Dear J,

    There are various explanations.
    1. As you can see from the BIC equation, the (implicit, in most cases) penalty for complexity involves N. Basically, with high N the predictions from H1 become more ambitious: if there is a true effect, with high N you expect large effect sizes in the sample. If you don't find these, your evidence may favor H0 (or favor H1 less then you might intuit at first glance).
    2. You can consider this a sequential experiment. First study yields BF = 4.2. Now if you do Bayesian updating, the posterior distribution after study 1 will be your prior for the analysis of study 2. The posterior distribution after study 1 is relatively optimistic. Now study 2 comes in, and the data are not as strong as the posterior from study 1 had you expect. In your case, the point null actually predicts a little better than the posterior based on study 1. Specifically, the BF for study 2 given study 1 is 1.6/4.2 = 0.38.

    Cheers,
    E.J.

Sign In or Register to comment.