Post-hoc corrections in Bayesian RM ANOVA

AnnaPsy · August 21

Hi everyone,

I have a question regarding post-hoc comparison in a Bayesian RM ANOVA.

I have a 2 × 2 × 2 × 2 design. Let’s say two of the factors are GROUP (between-subjects: Group 1 vs. Group 2) and Taste (within-subjects: Tase 1 vs. Taste 2). In my Bayesian ANOVA I found a significant GROUP × Taste interaction.

I would like to decompose this interaction in both directions: 1) Compare Taste1 vs. Taste2 within each group (Group 1 and Group 2). 2) Compare Group 1 vs. Group 2 within each level of Taste (Taste1 and Taste2).

This gives me four simple effects tests. My question is: in Bayesian analysis, do I need to apply any correction for multiple comparisons (as in Bonferroni/Holm´s with frequentist post-hoc tests), or can I simply report the Bayes factors for these four contrasts as planned follow-up tests to the interaction? Does anybody know any paper or reference?

Thanks a lot in advance for your help :)!

Best.

TarandeepKang · August 25

Hi @AnnaPsy

I will preface this by saying I'm not an expert in Bayesian statistics, especially what I would consider complex linear models such as this. However, I do think I can help you with this one so I shall do my best!Scroll down to the post hoc test panel under Bayesian repeated measures ANOVA and open it. Then drag in your variable interest (group?) To the right-hand side. Recall that in the ordinary Frquentist ANOVA the follow-up tests are essentially the required number of t-tests and then we apply some correction, depending on how conservative we are feeling about false-positive rates, taking into consideration power, et cetera.

There exists no post hoc test for the Bayesian formulation of the Anova. However what we can do is follow Westfall, P. H. (1997). Multiple testing of general contrasts using logical constraints and correlations. Journal of the American Statistical Association, 92(437), 299–306.) which Is what is suggested in Section 6 of

van den Bergh, D., van Doorn, J., Marsman, M., Draws, T., van Kesteren, E.-J., Derks, K., Dablander, F., Gronau, Q. F., Kucharský, Š., Gupta, A. R. K. N., Sarafoglou, A., Voelkel, J. G., Stefan, A., Ly, A., Hinne, M., Matzke, D., & Wagenmakers, E.-J. (2020). A tutorial on conducting and interpreting a Bayesian ANOVA in JASP. L'Année Psychologique, 120(1), 73–96. https://doi.org/10.3917/anpsy1.201.0073

You will then find adjusted posterior odds as described in the tutorial paper. This is the extent of post hoc testing that is currently possible, I think. I think the author team who are also the JASP developers do a good job of laying out the maths there in that section. Regardless, I will be happy to help you further if you need?

Best,

Tarandeep

NB, I don't know if you have found the analysis help file that is available in the software, you can activate it by clicking the little i? There is some stuff on this topic in there also.

EJ · September 12

Hello AnnaPsy,

Tarandeep is mostly correct. We do have Bayesian post-hoc tests in ANOVA though, and they follow the Westfall et al idea. You can read more about Bayesian corrections in a thesis of my former student and JASP programmer Tim de Jong:

https://osf.io/preprints/psyarxiv/s56mk_v1

TarandeepKang · September 15

Thanks for the clarification EJ!, I will give that thesis a read too!

Live and learn. :)

Howdy, Stranger!

Categories

Post-hoc corrections in Bayesian RM ANOVA

Comments

Howdy, Stranger!

Quick Links

Categories

Post-hoc corrections in Bayesian RM ANOVA

Comments