Wrong results from ANOVA post hoc

rhlamber

Hi there.

A couple of months ago I have asked JASP team to implement the post hoc analysis for the interaction of groups from a two way ANOVA.

I would like to start thanking the team for the implementation of this test.

Working with my data in JASP I found some weird p values resulted from the post hoc of the interaction of groups.

To compare, I ran the same results in SPSS and GraphPad Prism. The overall ANOVA results are ok, however, as I suspected, the results provided by JASP on post hoc comparisons were wrong (SPSS and Prism presented the same p value between them). I tried all tests available (Tukey, Scheffe, Bonferroni, Holm and Sidak), and all returned wrong values from JASP.

Am I doing something wrong?

Please see attached the tables returned from SPSS and JASP. Also, I am providing a csv file of the data used on these analysis (a created result only for example).

Looking forward for the team reply.

Thanks in advance.

Test.csv

SPSS vs JASP.docx

JohnnyB

Hi rhlamber,

From a quick look at the tables you included in your document, it seems that SPSS is performing fewer post hoc comparisons - it does not compare the different levels of group (sedentary vs exercised), whereas JASP does. This means that more comparisons are performed in JASP, which impacts the post hoc p-value corrections (note that in SPSS certain comparisons are repeated, but in reverse). JASP performs 15 tests in the first table, whereas SPSS only performed 6.

Kind regards,

Johnny

rhlamber

Hi Johnny.

Thank you very much for your reply!

I understood your point. SPSS do not compare all groups at same time like JASP. It can only compare in two stages (first by time and then by group). Considering this, the adjusted p has to be different (not wrong as I stated before... sorry about that).

Thanks again.

Best regards