#### Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

# Replication Bayes factors in R [BayesFactor package]

I would like to compute the Bayes factor for a replication study, as recommended here: https://link.springer.com/article/10.3758/s13428-018-1092-x and here: https://www.ncbi.nlm.nih.gov/pubmed/24867486. In a nutshell, I'd like to take the posterior distribution from Experiment #1 and use it as the prior for Experiment #2. Can this be done using the BayesFactor package in R, and if so, how?

• library(BayesFactor)

# say you have 2 data sets

iris_1 <- iris[1:75, ]

iris_2 <- iris[-(1:75), ]

# To get a replication BF you need:

## 1. BF of the first data set

BF1 <- lmBF(Sepal.Length ~ Sepal.Width + Petal.Length,

data = iris_1)

## 2. BF of the combined data sets

BF_total <- lmBF(Sepal.Length ~ Sepal.Width + Petal.Length,

data = rbind(iris_1,iris_2))

## 3. Divide the latter by the former:

extractBF(BF_total, logbf = F, onlybf = T) / extractBF(BF1, logbf = F, onlybf = T)

#> [1] 2.564315e+32

# (compare to)

lmBF(Sepal.Length ~ Sepal.Width + Petal.Length, data = iris_2)

#> Bayes factor analysis

#> --------------

#> [1] Sepal.Width + Petal.Length : 2.170426e+18 ±0%

#>

#> Against denominator:

#>  Intercept only

#> ---

#> Bayes factor type: BFlinearModel, JZS

• edited May 19

Thanks so much, this works perfectly!

One follow up question: I'm wondering if it would make sense to use this approach iteratively to compute a replication Bayes factor based on multiple previous experiments? So, assuming there are three experiments, the first two showing the same effect, and I wanted to use the (combined) posteriors from the first two experiments as a prior for the third, could I use...

# 3 data sets

iris_1 <- iris[1:50, ]

iris_2 <- iris[50:100, ]

iris_3 <- iris[-(1:50),]

## 1. BF of the first TWO data sets

BF_first_two <- lmBF(Sepal.Length ~ Sepal.Width + Petal.Length,

data = rbind(iris_1, iris_2))

## 2. BF of the combined data set from all three experiments

BF_total <- lmBF(Sepal.Length ~ Sepal.Width + Petal.Length,

data = rbind(iris_1, iris_2, iris_3))

## 3. Divide the latter by the former:

extractBF(BF_total, logbf = F, onlybf = T) / extractBF(BF_first_two, logbf = F, onlybf = T)

• Yes, this should generally work for any K datasets from replications - but note that all data must be from exact replications for this kind of analysis to make sense.

Good luck!

• @MSB I'm curious what you mean by "exact replication" in this context. I'm in a situation where Experiment 1 compared performance between two conditions (let's call them condition A and condition B), and Experiment 2 included the same conditions (and used the same paradigm / method of assessing performance / etc etc) PLUS one extra control condition (condition C). So you could say Experiment 2 was an exact replication, with the exception of one additional condition. Moreover the effect I was trying to replicate was still between conditions A and B.

Would this still satisfy the criteria for an exact replication (in the context of using replication BF)? If not, would it make sense to exclude data from the new condition, and simply compute the replication BF for the contrast between conditions A and B in Experiment 2?

• I don't think so - because the second model's parameters are not the same as the parameters of the first (it has one more). But @EJ would probably know best (:

• Well if you throw away condition C then everything is fine

E.J.

• Amazing, thank you both so much!