Same pairs of comparison in 2 separate 4x3 ANOVAs, but different Tukey's post-hoc p-values.
So I initially wanted to do a 4x3x3 ANOVA but decided to break it down into two 4x3 ANOVAs instead to get a better look at the data. The data used for the two ANOVAs are based on the similar set of data. But what puzzled me, is that during post-hoc testing, a different p-value was generated for the exact same pair of comparison. This is despite the fact that both ANOVAs are utilising the same factorial design.
The two large arrows here on the first figure points to the same mean difference to show that both comparisons are similar. But the subsequent CI, t and p values are different. My gut feel tells me that I may be making an amateurish mistake somewhere but I can't figure out what is it. Can anyone explain to me why is this happening?
As an extra sanity check, here on the descriptive stats shows that the individual means and SD for these comparisons, which are exactly the same.
Comments
I assume this is because the test is based on a pooled standard error, but I'll ask our expert to confirm this...
Cheers,
E.J.
Hi AaronAng,
I see that not all standard deviations are the same in your descriptives table, which will lead to different pooled standard deviations and standard errors, and therefore t and p values.
Kind regards
Johnny
Ah, thanks for the clarification Johnny!