Bayesian Statistics: Correcting for multiple testing?
recently, I found out that multiple testing can become a problem also with Bayesian statistics, if the tests are not independent. I have already collected data for my study and would like to ask you, if a correction for alpha error rates is necessary and if so, how to do that with JASP?
The design is the following:
Independent Variable (IV): Participants answer three different questionnaires. For analysis, we split them into two groups per questionnaire (high vs. low scores) based on a pre-defined cut-off. In sum, there are 6 groups (3 questionnaires x 2 subgroups).
Dependent Variable (DV): All participants complete 3 different tasks. Each task refers to one of the three questionnaires.
Then, for each of the 6 groups a Bayesian one-sample t-test of the respective DV against the expectation value under chance is performed. We considered each questionnaire (IV) plus the respective task (DV) as independent experiment with a Bayesian hypothesis for or against an effect.
Problem: The three questionnaires highly intercorrelate. This means, there is a high probability that the same participants are in more than one of the high-score subsamples. This is kind of a dependency, but does this influence the error rates in a Bayesian design or how can I correct for that?
Thank you very much for your support!