Bayesian replacement for MANOVA
Dear all,
Attached is a file showing an example of the data that I want to employ Bayesian inference on. So, the experiment has 3 conditions, and the participants in each condition has seen one of eight short video clips. They have then answered four likert scale questions (scale 1-10; M1-M4). They have also contributed with free text resulting in a percentage of relevant mentions, and one test, resulting in a percentage correct.
I would like to see if there are differences between the three different conditions. I would also like to narrow it down to see if condition 2 has a positive difference and conditon 3 has a negative difference in comparison to the control (condition 1).
What type of analysis would you do in Jasp. And which corrections would you recommend for the multiple tests?
I really appreciate your help.
Comments
There are multiple complications here. First you have both subjects and clips, so I think this calls for a crossed-random effects analysis. Then you have multiple dependent variables -- we don't have a Bayesian MANOVA yet, but maybe the mixed models section has one (I'll ask the team member who implemented this).
Then you have specific interests in some of the conditions -- maybe this can be handled with contrasts. Finally, your measurement scale is ordinal; this is usually ignored, but ideally it should be taken into account. My assessment is that if you want to do this well, you'd take a look at the Bayesian mixed model implementation in JASP; you might need to go further and work with brms in R. This is not a trivial analysis and if you want to do it neatly you might need some expert to dedicate time to this.
Cheers,
E.J.
Dear dmalm,
As EJ noted, a Bayesian MANOVA would be the most appropriate analysis given your description. Unfortunately, the Mixed Models module does not provide multivariate models (and I haven't seen them in any application outside of textbooks yet).
You mention that you have 4 items, are they related and do they form a scale? If so, you might want to compute a mean score and model that as your dependent (univariate) variable. The Bayesian Repeated Measures ANOVA should then allow you to perform the analysis (although you might need to use the frequentist version for test the order restrictions). You can also estimate the parameters with the Mixed Models module.
If they are not related, the easiest approach would be to analyze them separately. There are many ways for dealing with multiple tests. The most important thing is, in my opinion, to clearly disclose and report all the tests you performed so readers have a clear picture of what has been done. Otherwise, a common approach is to adjust prior model probabilities to reflect the multiple hypotheses being tested.
Dear E.J., and František,
thank you for taking the time to answer my question. I do have 4 items, I would however treat two of them as explorational, not including them in the main analysis. What I would like to do is perform an unbiased estimate on a 3 (condition; baseline, pos, neg) x 2 (true or fals) x 2 (probable / improbable) design. Possibly, it would be better to use the condition as a covariate.
If I would do a Bayesian anova and anlysing the 2 x 2 answers (likert scale, related and answered by all participants), how would I adjust for multiple testing?
I've used the mulrank and cmanova function in R to do robust frequentistic manova, using the 2x2 to form 4 groups. It's significant. However, I would prefer to use a Bayesian equivalent measure.
I sincerely appreciate your support and help on this.
Best,
D
I'm with Frantisek here: just be transparent about the number of tests you did.
E.J.
Will do! Thx