How to Perform a 2x2x2 Mixed ANOVA Analysis?
Hello all, I am still very new to JASP, but as part of a research project I am working on, I need to do some deeper data analysis. I am however having trouble learning how to perform the specific test I need, a three-way mixed ANOVA with one between factor and two within factors.
Here is a simple example of what that data would look like:
The analysis should have "Sex" as the between factor and "Right/Left_HighLow" as the two within factors. Friendliness is the Dependent variable.
Any help would be appreciated. Thank you.
Comments
An update:
I have figured out mostly how to do it using the following method. There are some strange behaviors though...
The first thing I had to figure out was that I needed to format my data differently by separating the dependent column into one for each of the four conditions so that it looked like this:
This then allowed me to use the "Repeated Measures ANOVA" (despite the conditions being independent from one another) to first create two factors "Right_LowHigh" and "Left_LowHigh" each with two levels "Right/Left Low/High." I then dragged in the corresponding friendliness scores into each combination in the "within" box. I then put "Sex" in the "between" box.
This allowed me to perform simple and conditional ANOVA on the set as I desired and I could verify this with someone else who performed these tests using SPSS.
There are two anomalies, though. The first is that in Model -> Between Subjects Components, that the combination model term appears when creating the factors, but then disappears and cannot be replaced if it is dragged out of the "Model terms" list. In other words, it never appears in the "Between Subjects Components" list.
Secondly, I was able to verify the conditional post hoc tests for all two factor (for example Sex * Right_LowHigh) tests against my co-worker who used SPSS, but once it came to the three term tests (Sex * RightLowHigh * Left_LowHigh), our results differed. Is it possible this is a bug?
Again, thank you for any help that comes.
Once you've specified the factors, you should not do anything to the model. If I remember correctly, if you have deleted any interaction terms from the 'model' (which you generally should not do), if you ctrl cliçk to highlight multiple factor and then drag that highlighted set in, the interaction terms will be restored (if that doesn't work, just start all of over again).
It's expected that adding (or removing) factors will change the post hoc test results, since the data are being divided up differently.
R
Also, the last time a checked, spss could only do post-hocs relevant to main effects. For decades it has not been able to do post hocs relative to interactions.
R
@andersony3k , Thank you for your help explaining the first anomaly I noticed. I have no reason to change the model in the middle of the analysis, it just seemed strange to me that this place in the GUI would have an item that once moved from the in-use box does not appear in ready-to-use box like in all the other parts of the GUI. If this comes up again, I will try your method of highlighting the two factors I want to see the interaction of.
That's one of my concerns dealt with, the second (where I am able to reproduce the two-factor post-hoc results that SPSS can but not the three-factor). I don't know which results are true and one (SPSS) shows there are some significant three-factor results but JASP does not. It seems strange to me that I would be able to get the exact same values for both the one/two-factor tests but then different ones for the three-factor.
Can you provide screen shots showing the disparate post hoc results?
R
Here is an example of one of the two-factor post-hocs and one of the three-factor ones. All the other one/two-factor tests match up between the two tools.
I apologize for the Japanese on the SPSS results. Some notes on the naming. For these analyses, Gender = Sex, 1 = Low, and 2 = High.
Thank you again for your help. I hope these more specific examples can help you diagnose the issue.
Could you possibly post the jasp file (zipped)? Or if not, for privacy reasons, a version of the jasp file that includes arbitrary or random values if the dependent measure instead of the real values?
R
I'll also have our expert take a look at this.
EJ
Hi @Aidan,
For post hoc tests, JASP conditions on max 1 factor, while SPSS creates nested conditioning. Here, that means the two post hoc tables show different analyses:
Comparing 4 marginal means to each other gives 4*3/2 = 6 comparisons (see JASP footnote), which means the bonferoni p-value gets multiplied by 6.
The SPSS table compares 2 marginal means, which gives 1 comparison, so the Bonferroni p-value gets multiplied by 1. This explains why some of the comparisons that are identical between the tables have the same mean difference, but their Bonferroni p-value is 6 times larger in the JASP output.
Based on either table though, I would conclude there is no evidence that any of the groups differ from each other - the effect is too weak and/or the group sizes too small. Both softwares are a bit rigid in how they do their interaction post hoc tests - at least for JASP I can say that we will add the option to nest further or not in the near future.
As for the first concern, I'm not sure I follow. When you have a between-subjects factor it shows up in the "Between subjects components" box in the "Model" tab. When you include a between subjects factor, automatically any interaction effects between that factor and all within-subjects model terms will be added. Removing the model term from the box omits it from the model entirely (both main effect and interaction effect). I'm not sure what you refer to with "disappearing" - could you maybe include a screenshot? It could also be that we are using different JASP versions - are you using the latest version, 0.95.4?
For some extra illustration of (mixed) ANOVA, you can take a look at this data example, or consult the JASP video library (only RM ANOVA, not mixed for now).
Also thanks a lot for helping out earlier @andersony3k =)
Kind regards,
Johnny
Dear all,
To be clear, I'd emphasize that SPSS has never been adequate for post-hoc testing in ANOVA models containing interaction terms. Each post hoc-test in SPSS tests the effect across two levels of a single variable, but will not test across two conditions defined by thin interaction of two variables (and will not do post-hocs that are redundant with the ANOVA main-effect results). So I don't recommend SPSS for ANOVA since it can't produce complete post-hoc results. Moreover, one cannot use SPSS to check JASP's post-hoc results since SPSS doesn't produce all of the relevant post-hoc test results.
R
Thank you all for your continued help. I will try my best to respond to all of your points here.
@andersony3k You will find a JASP file attached to this message that has semi-arbitrary values and I have set up the analysis in the same way that I am working with with my data. I hope that this provides the information you are interested in seeing.
- - -
@EJ Thank you for directing someone to take a look.
- - -
@JohnnyB Thank you for this explanation on the differences between JASP's and SPSS's ways of comparing combinations. I had not noticed that 6x pattern before.
So, it sounds like since the JASP comparisons are more rigorous, I should likely consider those differences (or lack thereof) as the proper evidence of effects. Or, multiply the p-values of the SPSS by 6 potentially. I am still learning how to actually perform these kinds of tests so am still somewhat naïve to which tests/tools are best for the job. I appreciate your help.
About the technical concern, here is a demonstration of what I mean. You can see that unlike other parts of the UI where dragging something off of the right panel returns it to the left, here dragging it off of the right just makes the item disappear. This then requires the user to remake one of the factors so the term is returned to the right list.
- - -
@andersony3k Thank you for these details. I will have to look up some of these terms like "thin interaction" since I feel that is a term of art, rather than a simple description of weak interactions.
It appears that between both you and @JohnnyB 's descriptions of the differences between JASP and SPSS, that it may be wise for me to use the JASP post-hoc analysis results for my work. I will do some more thinking and talking with my peers.
- - -
Thank you all again for your time and support.
Edit: I've done some more thinking and talking with a peer and have found this discussion about the differences between JASP and SPSS challenging, but very helpful for me understanding what is best for me, my research, and my communication with others about my results. So thank you all for engaging with me in a way that has aided me in learning more about my data and what is appropriate for this case.