Extracting BFs from anovaBF or any Formal class BFBayesFactor object

NaomiHavron

Hi,

I am using BayesFactor to run an ANOVA:

model<- anovaBF(dif ~ cond * exp, data=df)

The output appears in the console, but if I want to save it as an object, then the object class for this object is a "Formal class BFBayesFactor". What I need is to extract only the BF10 for a given line, so that I could save this as a row in a new dataframe (this is because I want to run the model on 1000 simulated data sets and see how many of them show an effect of a given size...).

I can get one of the slots of the Formal class object by using @ (e.g. model@bayesFactor). However, the values in the bayesFactor slot do not correspond to the values printed in the console.

How do I get access to the values that are printed in the console?

Thanks in advance,

Naomi

MSB

You can use as.vector():

library(BayesFactor)

data(puzzles)

result <- anovaBF(RT ~ shape*color + ID, data = puzzles,
         whichRandom = "ID", progress = FALSE)

result
#> Bayes factor analysis
#> --------------
#> [1] shape + ID                       : 2.840104 ±0.84%
#> [2] color + ID                       : 2.841925 ±1.48%
#> [3] shape + color + ID               : 11.43469 ±1.23%
#> [4] shape + color + shape:color + ID : 4.873394 ±10.38%
#> 
#> Against denominator:
#>  RT ~ ID 
#> ---
#> Bayes factor type: BFlinearModel, JZS

as.vector(result)
#>            shape + ID            color + ID 
#>             2.840104             2.841925 
#>        shape + color + ID shape + color + shape:color + ID 
#>            11.434689             4.873394

NaomiHavron

This works, thank you for such a swift and helpful response!

___________________________

Now maybe you could also answer a more conceptual question?

My own dataset gives inconclusive results for my effect of interest (BF10 = ~2).

A reviewer asked how many participants I would have to add to cross my desired strength of evidence (BF10>3), and suggested I run simulations. I thought it makes sense to create a larger sample by creating a fake dataset with the same betas as my actual regression (which I obtained from a non-bayesian analysis):

# Set coefficients
alpha = -1.3091
beta1 = .0556
beta2 = -1.8874
beta3 = 6.0730

# Generate 100 trials
A = c(rep(c(0), 50), rep(c(1), 50)) # '0' 100 times, '1' 100 times
B = rep(c(rep(c(0), 25), rep(c(1), 25)), 2) # '0'x50, '1'x50, '0'x50, '1'x50
e = rnorm(100, 0, sd=1) # Random noise, with standard deviation of 1

# Generate data using the regression equation
y = alpha + beta1*A + beta2*B + beta3*A*B + e

# Join the variables in a data frame
data = data.frame(cbind(A, B, y))

# Fit a regression
model = generalTestBF(y ~ A*B, data=data)

Now, if I increase the number of rows in my fake data set (let's say from 100 to 300), the strength of evidence should logically increase, am I right? Or maybe it doesn't have to? Because when I do that I get more or less the same result (~700 of my fake datasets give a result around BF10~3, and the rest give a BF~2). Maybe it's my NHT logic at work and it doesn't work this way for a Bayesian analysis?

MSB

Generally, BFs are suppose to converge to the "truth" as N increases. So I depends what "truth" your betas are closer to, I guess.

For determining N, you might also be interested in https://github.com/nicebread/BFDA

NaomiHavron

Well, according to the references on the BDFA page (thank you!!) the evidence should become more conclusive as I increase sample size (either towards H1 or H0, but it should still change). So I must be doing something wrong somewhere.

EJ

Hi Naomi,

In general, asymptotically, and on average (!), increasing sample size leads to more conclusive BFs. But for a specific data set, a BF might become less compelling as N grows (up to a point). Also, the BFs may increase (again, on average) only slowly, depending on the details of the model specification -- if the data show a whopper of an effect, evidence will initially increase steeply, but if the effect is modest the change is evidence may only be noticeable for a large increase in sample size.

Cheers

E.J.

NaomiHavron

Thanks!