ANOVA Warning

HTIDCam

Hello,

Firstly, thank you for your work in developing JASP, it is a brilliant piece of software that I am sure will go a long way!

However, I am having trouble performing post hoc analysis during an ANOVA. Very much like the example for vitamin vs orange juice supplementation at different dosages, I want to look at changes in muscle mass (dependent variable) between males and females at different ages (fixed factors). However, when I select these for my ANOVA and post hoc, I get a warning underneath the output saying "Singular fit encountered; one or more variables are a linear combination of other predictor variables."

I am not sure if this error is arising due to how I have set up the csv file for import into JASP (see attached), but it would appear that the imported format reciprocates the example, so I am sure why I am getting this error message.

Any help would be greatly appreciated!

EJ

Dear HTIDCam,

It might help us if you add the .jasp file and/or a screenshot of the error message. I will bring this to the attention of one of our team members. Usually Johnny would handle this but he's on vacation, so you might have to wait a few more days.

Cheers,
E.J.

HTIDCam

Hi E.J.

Thank you for the swift response.

not a problem, see attached the .jasp file with the ANOVA performed. For sake of ease, I have also attached the example ANOVA .jasp file with the ANOVA performed.

I did not have a "30 week" old age group for the females but did for the males. Will JASP not perform post hoc if there is not a matching level (ie age) for another group? The reason I ask is because I removed the 30 week old males from my analysis and the post hoc worked, however I need that age in the analysis. I seem to have no problems in SPSS as this seems to "ignore" the fact i don't have a 30 week old female group, however I would much rather use JASP.

Kind regards,

Cameron

JohnnyB

Hi Cameron,
Thanks for the detailed response and including your jasp-files! I'm currently taking a look at what goes wrong under the hood, and it indeed seems like the lack of 30 week old females is causing the trouble here. If a case like this occurs, JASP gives a warning to indicate the lack of one or more groups, but it does still output the ANOVA table. It does not compute post-hoc tables anymore if this singular case occurs, so that's why they remain blanc. A work around for this is to exclude the time variable, and compute post-hoc tests for gender, and then the other way around for time, since the post-hoc tests are only taking one variable into account and the lack of one combination of levels of factors should not matter.
I am going to look into this issue in more detail, and try to make it so that the post-hoc tables will still get made if there is a dataset similar to yours!

Cheers,
Johnny

edit: see also the attached pdf I made with your post-hoc tests.

VC

Hi, I am having a similar issue whereby the two-way ANOVA results for JASP and SPSS do not match at all. I get the same warning about empty cells. It is my understanding that SPSS uses a formula that adjusts for unequal cell sizes, but is this true for JASP? My assumption is yes but I have no idea why the results are so different. I would like to use Bayesian ANOVA, so I need to rectify this discrepancy in results. I have triple checked the file and data and know they are correct.

I copy the JASP output here and the SPSS output.

Thanks!

JohnnyB

Hi VC,
After an intense investigation, I am not able to pinpoint exactly what SPSS does, sadly enough. If you have any idea, then I would greatly appreciate being pointed in the right direction
JASP uses the car package in R - so I'm able to replicate JASP's results:

mydat <- read.csv("/home/johnny/Downloads/anovaCheckJASP.csv")
mylm <- lm(mydat$Education ~ as.factor(mydat$P_Agegroup) * as.factor(mydat$P_Educ_3))
Anova(mylm, type = 3, singular.ok = T )

Note that what is causing the discrepancy in results is the presence of a missing cell in your design matrix. Type 3 ANOVA's are particularly sensitive to this, which is why JASP (and R-packages) issue warnings when performing a type 3 ANOVA with missing cells.
In R, however, you can overrule this warning with the argument singular.ok, so that the sums of squares and other statistics are still calculated. They will not be very informative (hence the warning message); to quote this informative document on the different types:
``for ANOVA designs with missing cells, Type III sums of squares generally do not test hypotheses about least squares means, but instead test hypotheses that are complex functions of the patterns of missing cells in higher-order containing interactions and that are ordinarily not meaningful. "

There is light at the end of your tunnel though! In the Bayesian framework, we can do away with these types of sums of squares, so your Bayesian result will be (even more!) straightforward.
Please let me know if anything needs clarifying!
Cheers
Johnny