Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

interpreting error percentage

Hi all,

I would like to consult this forum, and check that I understand how to interpret the error percentage presented with the Bayes factor values.

If I understand the JASP manual correctly, the error percentage is the percentage to which the BF value can increase or decrease due to Monte-Carlo simulation noise randomisation. So, for example, if I have a BF10 value of 14.150 and an 28.539% error percentage, it means that upon recomputation, the BF10 can fluctuate approximately between 10 to 18 (14.150*0.285= approximately 4.05).

Is that a correct interpretation?

Thank you very much,

Lior

Comments

  • Hi Lior,

    Hmm I'm not so sure. You are right about the 28% corresponding to a BF change of 4, but I would have guessed it indicates that the BF10 can fluctuate from approximately 12 to 16. I'll double-check...

    Cheers,

    E.J.

  • Hi Lior,

    This is an excerpt from our Bayesian guidelines article (https://psyarxiv.com/yqxfr), where we discuss the error percentage for ANOVA:

    "For some analyses, the results are based on a numerical algorithm, such as Markov chain Monte Carlo (MCMC), which yields an error percentage. If applicable and available, the error percentage ought to be reported too, to indicate the numeric robustness of the result. Lower values of the error percentage indicate greater numerical stability of the result. We generally recommend error percentages below 20\% as acceptable. A 20\% change in the Bayes factor will result in one making the same qualitative conclusions. However, this threshold naturally increases with the magnitude of the Bayes factor.

    For instance, a Bayes factor of 10 with a 50\% error percentage could be expected to fluctuate between 5 and 15 upon recomputation. This could be considered a large change. However, with a Bayes factor of 1000 a 50\% reduction would still leave us with overwhelming evidence."


    So your interpretation is correct. You are of course free to use your own judgment for which percentages are acceptable, we just provide a bit of intuition for it. You could try running the analysis by increasing the number for "numerical accuracy" or by using more MCMC samples ("posterior samples").


    Kind regards,

    Johnny

  • Great, thank you so much!

Sign In or Register to comment.