Discrepancy between Bayesian and Frequentist Repeated Measures ANOVA

cames

Hi,

I am analyzing the effect of two different drugs on cognition. Clinically, we expect both drugs to have an effect, one possibly more than the other. Therefore, I used a 2x2 design with the within-subject factor "time" and the between-subject factor "treatment arm". The treatment arms differed in two other factors, which were included as covariates.

I'd like to use Bayes Statistics, but put the frequentist equivalent into additional material. Interestingly, the two analyses show different results: The RM ANOVA shows a significant if small interaction effect time * treatment arm (F=4.32, p=.04, partial eta² 0.06) but no simple effects of either factor. The Bayes RM ANOVA shows a very strong simple effect of time (BFincl = 1124.9), but no interaction effect.

I don't think the results are necessarily incompatible with our clinical observations but I'm confused by the difference.

I checked the forum for other discussions and found violated assumptions as possible explanation. Sphericity is met as the RM factor has only two levels, but Levene's test for homogeneity is significant. Trying to use the nonparametric addition (and thank you for that!) gave me the error message "Specified ANOVA design is not balanced". I am not sure what this means.

Are there any other possible reasons for the differences in the results? Is there some other procedure I should try?

Thank you very much!

Bayesian Repeated Measures ANOVA.pdf

JohnnyB

Hi Cames,

For the nonparametric ANOVA, when specifying a grouping factor, it needs to be at least an incomplete balanced design in order to run the Friedman or Durbin test. Such a design has the following properties (see https://en.wikipedia.org/wiki/Durbin_test):

1. A balanced incomplete block design has the following properties: Every block contains k experimental units.
2. Every treatment appears in r blocks.
3. Every treatment appears with every other treatment an equal number of times.

If you can provide a JASP file of your analysis, I could look a bit more closely into the results and the discrepancies between the parametric analyses.

Kind regards,

Johnny

cames

Hi Johnny,

That makes sense :)

Thank you for your help! The JASP file is attached. I confess that the posterior plots are puzzling to me, but maybe they're the answer?

Kind regards,

Cames

cames

oh sorry, the request to attach a file failed with status code 500

cames

ANON_adjunctive ZNS TPM_short.jasp

alright, so the full analysis was too big to attach even as a zip folder, but I think I included the most relevant analyses for my problem. Thank you for taking the time to look into it!