Testing assumptions for Bayesian Repeated Measure ANOVA

Comments

• EJ

  April 2018

  Hi Tom,

  The Bayesian ANOVA (it is really a linear mixed model, see the BayesFactor documentation) makes the same assumptions as the classical ANOVA. We just have not developed the Bayesian echoes for those assumption tests (yet). We will do this in the future, but until that's done you can take a pragmatic approach and use the frequentist tests. There is of course an immediate practical problem -- suppose you do want to correct, how would you do it? In the Bayesian framework, instead of issuing a correction you would apply a more complicated model that can account for the misspecification. But that is work for the future. In general, we hope to give the Bayesian ANOVA some more love in the future.

  Cheers,
  E.J.
• TooFred

  April 2018

  Dear E.J.,

  Many thanks for responding quick to my questions. Much appreciated. I have noted all that you said and will do as suggested. Please one other question: in terms of reporting effect size for the Bayesian ANOVA, how do I go about this? I know for the classical ANOVA, you can obtain partial eta squared, which is also an option in JASP when using the classical RM ANOVA. Lastly, the advanced options for the prior, do I leave the default values as they are or do I have to alter them? I understand the complexities with setting priors and that JASP uses default priors. And what do these represent: r-scale fixed effect, r-scale random effects, r-scale covariates?

  Thank you once again as I look forward to hearing from you.

  Cheers,

  Tom
• EJ

  April 2018

  Re effect size: this is not straightforward (I think). There's a paper with Maarten Marsman that is currently somewhere in the review system. We need to polish the Bayesian ANOVA anyway in order to show parameter estimates. We'll take the effect size issue on board then as well. For now, I'd just report the frequentist effect size measure.
  Re priors for ANOVA: I'd leave them as is. The r-scales refer to the width of the prior distribution on the relevant effects. You can check the Rouder and Morey 2012 JMP paper if you feel brave enough. Maybe the BayesFactor documentation offers help as well.
  Cheers,
  E.J.
• AceOfBayes

  August 2020

  Dear JASP/BayesFactor experts,

  I would like to switch from NHST and p values to Bayesian analysis and BFs. In this endeavour I encountered very similar questions like TooFred. I'm fairly new to JASP/BayesFactor.

  How do I test and possibly correct for violations of the test assumptions of Bayesian rm-ANOVA? According to the "Guide for Students" (Mark Goss-Sampson wit van Doorn and EJ) the assumptions are:

  a) DV and residuals should be approximately Gaussian.

  b) No outliers.

  c) Variance homogeneity across factor levels.

  Two questions regarding this:

  1. JASP offers a visual test of normality by providing a QQ-plot. How would I test for variance homogeneity? Would I have to use an old-fashioned frequentist Levene test or F test?
  2. What about sphericity? As far as I know, frequentist rm-ANOVA has some robustness against violations of normality assumptions. Sphericity violations, however, are said to lead to way too liberal statistical decisions. Is this assumption missing in the documentation? Because it is not the same as c).

  To follow up on TooFred's question, I would be curious to learn if the JASPers/BFers have made progress on ways to correct for sphericity violations given what EJ wrote two years ago :) In my experience sphericity violations are more the rule rather than the exception in "real-life" research.

  So it would really be a pity if I were forced to resort to NHST just because they have Greenhouse-Geisser or Huynh-Feldt corrections for such violations while the validity of Bayesian statistical models remains limited to tutorial data (or factorial designs with only binary factors - which is already good! Just quite a limitation.)

  Is it still recommended to check for sphericity using Mauchly's test (or - alternatively - calculate GG's epsilon) and to switch back to NHST if needed? Wouldn't for instance a Bayesian Friedman test be a promising alternative in this scenario, which currently is not available in JASP?

  Thanks & best,

  Michael
• EJ

  August 2020

  Hi Michael,

  Yes we made some progress (e.g., https://www.bayesianspectacles.org/preprint-default-bayes-factors-for-testing-the-inequality-of-several-population-variances/), but this has not yet resulted in changes to JASP, unfortunately. I do think it's important to be able to do ALL analyses within the Bayesian paradigm, so I'll bump this on our priority list.

  Cheers,

  E.J.
• AceOfBayes

  August 2020

  Dear E.J., dear JASPers,

  OK, sounds great! From having a brief look at it, it seems that the proposed method would extend JASP in that it provides a test for checking assumption c) that I mentioned above. At present I'm checking this using Levene's test. The Q-Q plot takes care of assumption a). (FYI: I am referring to the assumptions as mentioned in this manual on page 90.)

  If I understand correctly though, the new paper you mentioned does not test/correct for sphericity violations, an assumption that is not mentioned in the accompanying documentation.

  It has been my understanding that variance homogeneity across conditions was an assumption of (classical) ANOVA with only between-subject factors and that due to the nature of repeated-measures ANOVA this assumption turns into the sphericity assumption (commonly tested with Mauchly's test), which concerns the homogeneity of variances across condition differences.

  Perhaps my point doesn't even really affect JASP per se but rather the author of that manual: this sphericity assumption does not even show up there.

  Cheers,

  Michael
• Annie

  June 2022

  Dear all,

  I am currently facing a similar problem as TooFred.

  I did a Bayes rmANOVA in JASP with 2 within-subject factors and tested the assumption of sphericity with the frequentist approach.

  Unfortunately, the assumption is violated.

  Do somebody know how I can correct it/apply another model?

  Thanks in advance!

  
  

  Annie
• andersony3k

  June 2022

  It seems to me that if researchers are to report frequentist effect sizes to go along with their Bayesian 'repeated measures' analysis, those effect sizes should not come from a frequentist repeated-measures ANOVA. Instead, they should come from a frequentist linear mixed model. This is because the data on which the Bayesian "repeated-measures" ANOVA are based may be different than the data on which the frequentist repeated-measures ANOVA are based: The frequentist repeated-measures ANOVA will automatically drop participants with incomplete data, whereas the Bayesian "repeated-measures" ANOVA and the frequentist mixed linear model will not drop participants with incomplete data.

  R