Justifying two prior distributions

arran_reader

Hello!

I wanted to check my reasoning regarding the choice of prior(s) for a one-sided Bayesian paired t-test.

A previous experiment observed an effect of d=0.74. We performed a conceptually related experiment, though not a replication. Since this other experiment provided the most relevant guide for an effect, we decided to use this prior effect size as the mean for a normally distributed prior when examining our data. However, given that the previously observed effect could be much bigger than what we expected to observe in our own experiment, we also centered a normal prior on half this effect size (0.37). The SD of these priors was set as 0.19, such that the two priors were separated by ~2SD.

My thought here is that this approach allows us to test against both a previously observed effect (that could be inflated, or larger than we might reasonably expect in our study), as well as a smaller effect which also has little overlap with zero and the larger effect size.

Is this approach reasonable? I have more concern over the SDs than the positioning. BF10 was <0.333 in both of the analyses.

The experiment and results are reported in a preprint here (e.g., line 426): https://osf.io/uckhf/

Thanks,

Arran

EJ

Hi Arran,

This is reasonable. Also, this distribution is close to several others that we have elicited from experts. For instance, it is similar to the "Oosterwijk prior" (see the informed t-test paper by Quentin Gronau, Alexander Ly, and myself) and it is similar to a prior that was used for a multi-lab study on the effect of ego-depletion (from memory: a N(0.3, sd=0.15) prior truncated at zero, so one-sided; the results are not yet in print but have been presented at SPSP in Atlanta last year).

Cheers,
E.J.

arran_reader

Thanks for the quick and helpful response EJ!

I've also been reading your recent JASP guidelines preprint. I'll report the robustness of the default prior in addition, given that our choice of effect size is only guided by one previous example.