#### Welcome!

Or register to create a new account.

We'll use your information only for signing in to this forum.

Supported by

# Output for % error in BF package/JASP

Posts: 28

Hi,
I have screened multiple articles of the usual suspects (Rouder/Morey/Wagenmakers) but I could not find any reference where the column with % Error comes from. What does in mean exactly and how is it computed?

• Posts: 461

Richard, can you explain how this is computed exactly?
We will also mention this in our new paper on JASP which will be on my website soon.
Cheers,
E.J.

• Posts: 461

So Quentin kindly looked into the R code for the BayesFactor package. It seems that the following function is used to calculate the error for the cases where the package does not use the integrate function
(obtained from https://github.com/richarddmorey/BayesFactor/blob/df23df41bea16e45c4e766c9104cc2a0ecc86e64/pkg/BayesFactor/R/common.R):

propErrorEst = function(logX){
logX = logX[!is.na(logX)]
summaries = logSummaryStats(logX)
exp( ( summaries\$logVar - log(length(logX)) )/2 - summaries\$logMean)
}

So this seems like a coefficient of variation (i.e., SD/mean).
For the t-test, the relevant code features "err = exp(log(intgl[[2]]) - val)", where intgl[[2]] corresponds to an "estimate of the modulus of the absolute error" (from the integrate function documentation) and we think "val" corresponds to the log Bayes factor.

Cheers,
E.J.

• Posts: 28

Trying to make sense here...it's a measure of the quality of the MCMC process? How much the BF varies between different chains? Seems to me that it is not absolutely necessary to report it?!

• Posts: 461

As long as the % is reasonably small (and it almost always is) there's no reason to report it. For ANOVA it can sometimes be a little bigger, but there we added the option to collect more samples to decrease the %. So is a a measure of the quality of the approximation. Sometimes the approximation requires MCMC, sometimes we use something else.
Cheers,
E.J.

• Posts: 28

alright, thank you very much for the support!

• Posts: 15

Hi,

I have the problem that when I conduct a paired samples t-test my %error is >7.000e-8. My BF is therefore not really valid and should not be interpreted, right?

Best,
Markus

• Posts: 461

7e-8 is a really small number!
E.J.

Thanked by 1Markus