Different results with paired t-test and ANOVA post-hoc paired t-test

Max123

Dear Team,

I recently tried to reproduce some analyses my professor conducted in SPSS. I could reproduce the ANOVA, however the post-hoc paired t-tests gave minimally different results. I then reproduced the post-hoc t-tests in R, which gives identical results to SPSS. After further experimentation I noticed that also the t-test function in JASP gives the same results as SPSS/R.

Therefore, I´d like to know how the calculation of the paired t-tests in JASP is different when using the t-test function compared to doing ANOVA post-hoc t-tests.

The differences are quite small, but noticable, for example t = 27.28 vs t = 27.35.

Best,

Max

JohnnyB

Hi Max,

Can you include your data example perhaps? That way I can zoom in on the problem, but I can also take some guesses:

• Are you running a normal ANOVA or a RM ANOVA? In the case of RM ANOVA, are you pooling the error term? Doing so or not will lead to different results.
• Are you running an unbalanced design? If so, then the results of the post hoc test will differ from individual paired t-tests, because the post hoc test uses the estimated marginal means (see my blogpost here), whereas the t-tests do not.
• Is there any missing data? Because these might be dealt with differently.

Kind regards,

Johnny

Max123

Hi Johnny,

thank you very much for the offer. Unfortunately, we cannot share the .jasp file on the forum because it is too big. Do you have an Email address we can send a filesender message to?

1) It is a RM ANOVA. I tried out whether pooling the error term accounts for the difference, it does not.

2) The design is balanced.

3)There is no missing data.

Best,

Max