Post Hoc tests for interaction in Bayesian repeated-measures ANOVA
Hello!
I am using JAPS for my analyses, but I have two main elements that appear to be missing in the last version (JASP 0.17.2.1). But perhaps there is another way to do them.
- My first problem is that in non-bayesian repeated-measures ANOVA, the marginal means display some possible interactions, but not all (see print screen). For instance, I have a significant triple interaction category*perspective*PTSD, but it does not appear in the options for the marginal means.
- I wanted to run the same versions of the analyses but with the Bayesian version. However, although in the non-bayesian there is the possibility to compute contrast, this option does not appear in the bayesian version (see print screen attached). I know I can run the Bayesian version of independent sample t-tests and paired sample t-tests to overcome the fact that there is not the contrast option available. However, the issue is that I need to run this analyses with a covariate (Age), which is possible to introduce in the repeated-measure ANOVA (bayesian and frequentist), but this covariate can't be introduce (as far as I know) in the paired and independent sample t-tests.
Would you have solutions for these two points?
Thank you very much!
Emilie
Comments
I've forwarded this to our expert!
E.J.
Hi @EmilieC ,
Kind regards
Johnny
Thank you for your answer.
Regarding point n°2, I don't know if you have seen this other post, but it looks like there is an issue with models including covariates, and I don't know at the moment if I can reliably use JASP for my covariate analyses
https://forum.cogsci.nl/discussion/8677/incongruency-main-effects-vs-post-hoc-comparisons#latest
HI @EmilieC ,
Yes, that one is definitely related, but it's a feature, rather than a bug (and consistent with other analysis software). The blogpost I linked explains this property more clearly, including an example.
Cheers
Johnny
Thank you for your answer,
I have post hoc comparisons for repeated-measures ANOVA, and I am asked to add the associated Bayes Factors. I had consider two options, which apparently are not reliable:
1) If I understand correctly, the Bayesian version will not provide reliable results, as not calculated on the marginal means (at the post hoc option only offers some main effects, not the contrasts for the interactions).
2) If I conduct Bayesian paired comparisons, same issue, as not on marginal means.
So at the moment, there is no proper correspondance between Post Hoc comparisons with the frequentist approach and with the Bayesian approach in JASP? Or is there a third option I forgot to consider that would provide me with these BF?
Thank you
@EmilieC I definitely would call myself a non-expert in Bayesian analysis, especially concerning complex linear models. Of course, even non-experts like myself need sometimes need to make scientific/statistical judgments that go a bit beyond our expertise. It is my judgment that, relative to frequentist models, simple Bayesian models (e.g.,Bayesian t tests) tend to be more extensively worked-out than complex (e.g., ANCOVA) Bayesian models. Therefore, I am more to rely on a Bayesian t test than a Bayesian ANOVA.
Here's my suggestion for some work that someone should do (maybe even get a dissertation out of it?): Figure out a way to generate not just a set of model estimates (estimated marginal means, estimated regression slopes and intercepts, etc.), and not just a set of observed deviations from the model (e.g., residuals), but instead, a *full estimate of what each data point in the data set would be* if the model were correct.
I imagine that such a "full estimate could be obtained simply by adding random noise to the estimated model parameters, except that the whole process would need to be iterated multiple times (to avoid a situation in which the result depends what particular random deviates happen to be added to the model parameter).
Because the full estimate would be a complete data set, it would then be straightforward to conduct Bayesian post-hoc or planned comparisons: Just conduct bunch of individual Bayesian t tests on the desired subsets of the estimated (not actual) data set.
R
Hi @EmilieC
If your design is balanced (also in terms of continuous covariates), then the marginal means will be at least highly similar to the descriptive means, so then the paired t-tests will give an accurate idea of the marginal means differences. It's a bit hard to say without having your specific results/data, so I can only give general guidelines here.
@andersony3k that's a great technique indeed, and is known as "posterior predictive check" - where you generate data under a certain model and see how well they match the observed data. So nice intuition ;-)
Kind regards
Johnny