Welcome!

Sign in with your CogSci, Facebook, Google, or Twitter account.

Or register to create a new account.

We'll use your information only for signing in to this forum.

Supported by

Output for % error in BF package/JASP

Hi,
I have screened multiple articles of the usual suspects (Rouder/Morey/Wagenmakers) ;) but I could not find any reference where the column with % Error comes from. What does in mean exactly and how is it computed?

Thanks for your answers in advance!

Comments

  • EJEJ Posts: 201

    Great question. Let's ask Richard.
    Richard, can you explain how this is computed exactly?
    We will also mention this in our new paper on JASP which will be on my website soon.
    Cheers,
    E.J.

  • EJEJ Posts: 201

    So Quentin kindly looked into the R code for the BayesFactor package. It seems that the following function is used to calculate the error for the cases where the package does not use the integrate function
    (obtained from https://github.com/richarddmorey/BayesFactor/blob/df23df41bea16e45c4e766c9104cc2a0ecc86e64/pkg/BayesFactor/R/common.R):

    propErrorEst = function(logX){
    logX = logX[!is.na(logX)]
    summaries = logSummaryStats(logX)
    exp( ( summaries$logVar - log(length(logX)) )/2 - summaries$logMean)
    }

    So this seems like a coefficient of variation (i.e., SD/mean).
    For the t-test, the relevant code features "err = exp(log(intgl[[2]]) - val)", where intgl[[2]] corresponds to an "estimate of the modulus of the absolute error" (from the integrate function documentation) and we think "val" corresponds to the log Bayes factor.

    Cheers,
    E.J.

  • duplex_duplex_ Posts: 13

    Trying to make sense here...it's a measure of the quality of the MCMC process? How much the BF varies between different chains? Seems to me that it is not absolutely necessary to report it?!

  • EJEJ Posts: 201

    As long as the % is reasonably small (and it almost always is) there's no reason to report it. For ANOVA it can sometimes be a little bigger, but there we added the option to collect more samples to decrease the %. So is a a measure of the quality of the approximation. Sometimes the approximation requires MCMC, sometimes we use something else.
    Cheers,
    E.J.

  • duplex_duplex_ Posts: 13

    alright, thank you very much for the support!

  • MarkusMarkus Posts: 15

    Hi,

    I have the problem that when I conduct a paired samples t-test my %error is >7.000e-8. My BF is therefore not really valid and should not be interpreted, right?

    Best,
    Markus

  • EJEJ Posts: 201

    7e-8 is a really small number!
    E.J.

    Thanked by 1Markus
Sign In or Register to comment.