Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Output for % error in BF package/JASP

Hi,
I have screened multiple articles of the usual suspects (Rouder/Morey/Wagenmakers) ;) but I could not find any reference where the column with % Error comes from. What does in mean exactly and how is it computed?

Thanks for your answers in advance!

Comments

  • Great question. Let's ask Richard.
    Richard, can you explain how this is computed exactly?
    We will also mention this in our new paper on JASP which will be on my website soon.
    Cheers,
    E.J.

  • So Quentin kindly looked into the R code for the BayesFactor package. It seems that the following function is used to calculate the error for the cases where the package does not use the integrate function
    (obtained from https://github.com/richarddmorey/BayesFactor/blob/df23df41bea16e45c4e766c9104cc2a0ecc86e64/pkg/BayesFactor/R/common.R):

    propErrorEst = function(logX){
    logX = logX[!is.na(logX)]
    summaries = logSummaryStats(logX)
    exp( ( summaries$logVar - log(length(logX)) )/2 - summaries$logMean)
    }

    So this seems like a coefficient of variation (i.e., SD/mean).
    For the t-test, the relevant code features "err = exp(log(intgl[[2]]) - val)", where intgl[[2]] corresponds to an "estimate of the modulus of the absolute error" (from the integrate function documentation) and we think "val" corresponds to the log Bayes factor.

    Cheers,
    E.J.

  • Trying to make sense here...it's a measure of the quality of the MCMC process? How much the BF varies between different chains? Seems to me that it is not absolutely necessary to report it?!

  • As long as the % is reasonably small (and it almost always is) there's no reason to report it. For ANOVA it can sometimes be a little bigger, but there we added the option to collect more samples to decrease the %. So is a a measure of the quality of the approximation. Sometimes the approximation requires MCMC, sometimes we use something else.
    Cheers,
    E.J.

  • alright, thank you very much for the support!

  • Hi,

    I have the problem that when I conduct a paired samples t-test my %error is >7.000e-8. My BF is therefore not really valid and should not be interpreted, right?

    Best,
    Markus

  • 7e-8 is a really small number!
    E.J.

Sign In or Register to comment.