Question model information linear mixed models JASP
We have been using JASP to do linear mixed models, but we were surprised to see that the results are different from when we run a linear mixed model with SPSS or R (UN or CS). It may be that the model is slightly different.
I was wondering if you could give me some more details on the linear mixed models in JASP:
- What covariance matrix does the model use?
- Can we compare the model fit (AIC/BIC) in JASP with the model fit in R/SPSS (if the dependent variable and fixed and random effects are the same)?
Many thanks for your time in advance.
Comments
my guess is that you need to look at Model -> Random effects:
By default JASP has the very bad default behavior of adding all possible random effects, whereas you specify those manually in SPSS or R (and probably rarely specify them all)
But is this bad behavior? What alternative procedure would you recommend? Not give any output until these effects are specified as fixed or random?
I would do random intercept personally
it's bad because it rarely runs (e.g. if you reproduce the example in the data library from scratch by going into mixed -> linear, it throws an error until you remove all random effects but the intercept, as in the example), and because that's rarely what the user wants to do
it's like JASP is taking the approach of model pruning rather than model building, but it rarely runs out of the box because of this, and not everyone is savvy enough to know what the error means...
Yes but we have to do something.
random intercept
So set everything else to fixed by default? But this seems to go against the popular mantra to "keep it maximal"
There is a lot of discussion in the literature how to properly specify random effects structure. However, one thing that majority of people agree on is that random intercept only models are suboptimal as they lead to poor coverage and inflated type I error rates. We therefore avoid using them as the default option.
We decided to use of the 'keep it maximal' recommendation i.e., fiting as complex random effects as possible. Of course, models should be simplified if they fail to converge (which is indicated by a warning message).
(We also automatically drop random factors for fixed effect predictors that do not vary between subjects, i.e., between subject factors, but that is also indicated in the messages)
Cheers,
Frantisek
Thanks for the answers. Do you have references that support the LMM model in JASP?
The idea of "keeping it maximal" as much as justified by the design, comes from this paper.
Barr DJ, Levy R, Scheepers C, Tily HJ. Random effects structure for confirmatory hypothesis testing: Keep it maximal. J Mem Lang. 2013 Apr;68(3):10.1016/j.jml.2012.11.001. doi: 10.1016/j.jml.2012.11.001. PMID: 24403724; PMCID: PMC3881361.
I hope this helps!
PS, I'm not a developer or a member of the team, just another JASP user