RM ANOVA different results in JASP and R.

albapy · February 2023

Dear JASP team,

I am running a RM three way ANOVA in R using:

anova_test(data, dv=A, wid=subject, within=(c(B,C,D))

This gives me slightly different results for some factors than running a RM in JASP e.g

Factor A: F= -1.02e-12 (on R) vs. F=2.878 x 10^-5 (JASP)

but for other factors it gives the same result e.g

Factor C: F= 1.7636 e+01 (on R) vs. F = 17.636 (on JASP)

Why is that so? some of the interaction F values are exactly the same whereas others differ.

Then, as I wanted to follow up on a significant A X C interaction running the following on R:

data %>%

group_by(A,C,subject) %>%

get_summary_stats(values, type='mean_ci') %>%

group_by(A) %>%

pairwise_t_test(mean ~ C, paired=TRUE)

This output gives the same as grouping my variables and averaging them and just running a paired t-test on JASP. However, it does not output the same as if I use the post-hoc default on JASP as a follow-up.

Why these differences?

Thanks!

patc3 · March 2023

Factor A: F= -1.02e-12 (on R) vs. F=2.878 x 10^-5 (JASP)

Hmm negative F-test isn't even possible...

JTPetter · March 2023

Hi there,

I would like to investigate this issue further, would you mind sharing the data here?

Best,

Jonas (JASP-team)

albapy · March 2023

Hi Jonas and patc3 thanks for your comments, I had not even noticed the negative distribution.

As I was prepping the files to share, I solved the issue.

It seems like there is something wrong with this particular file when writing it with matlab with:

writetable(file,'name.csv')

If I just open the file on excel and re-save it as a .csv now the R code and JASP give the same results.

Any ideas as to why this may have happened / suggestions on how to avoid it? It's very odd! / Does not happen with any of the other files I am saving in the exact same way.

Thanks very much

albapy · March 2023

Sorry, the above fixes the ANOVA results.

However, for the post-hoc t-tests, how is JASP calculating the post-hoc tests for an interaction vs. doing a paired samples t-test for the conditions of interest (averaging across the other factors)?

Both approaches still give slightly different t-values (but not sure if may be due to rounding errors or slightly different computations).

Still happy to share the data if still necessary.

Thanks very much

JohnnyB · April 2023

Hi @albapy ,

Sorry for the late reply. You can read about this behavior in this recent discussion: https://forum.cogsci.nl/discussion/comment/27509 and linked blogpost: https://jasp-stats.org/2020/04/14/the-wonderful-world-of-marginal-means/

Basically, the marginal means (which are estimated based on the specified model) can differ from the observed group means. Since these are used for the followup analyses (contrasts/posthoc), the results can differ.

Cheers

Johnny

Howdy, Stranger!

Categories

RM ANOVA different results in JASP and R.

Comments

Howdy, Stranger!

Quick Links

Categories

RM ANOVA different results in JASP and R.

Comments