interpreting error percentage

Comments

• EJ

  September 2019

  Hi Lior,

  Hmm I'm not so sure. You are right about the 28% corresponding to a BF change of 4, but I would have guessed it indicates that the BF10 can fluctuate from approximately 12 to 16. I'll double-check...

  Cheers,

  E.J.
• JohnnyB

  September 2019

  Hi Lior,

  This is an excerpt from our Bayesian guidelines article (https://psyarxiv.com/yqxfr), where we discuss the error percentage for ANOVA:

  "For some analyses, the results are based on a numerical algorithm, such as Markov chain Monte Carlo (MCMC), which yields an error percentage. If applicable and available, the error percentage ought to be reported too, to indicate the numeric robustness of the result. Lower values of the error percentage indicate greater numerical stability of the result. We generally recommend error percentages below 20\% as acceptable. A 20\% change in the Bayes factor will result in one making the same qualitative conclusions. However, this threshold naturally increases with the magnitude of the Bayes factor.

  For instance, a Bayes factor of 10 with a 50\% error percentage could be expected to fluctuate between 5 and 15 upon recomputation. This could be considered a large change. However, with a Bayes factor of 1000 a 50\% reduction would still leave us with overwhelming evidence."

  
  

  So your interpretation is correct. You are of course free to use your own judgment for which percentages are acceptable, we just provide a bit of intuition for it. You could try running the analysis by increasing the number for "numerical accuracy" or by using more MCMC samples ("posterior samples").

  
  

  Kind regards,

  Johnny
• LiorAbramson

  October 2019

  Great, thank you so much!