GLMM in JASP
Dear forum users, I am trying my best at a GLMM in JASP. Unfortunately I get a warning.
Warning: Numerical problems with the maximum-likelihood estimate (e.g., gradients too large). This may indicate that the specified random effects parameters (random intercepts and random slopes) cannot be estimated from the available data. Consider carefully reducing the random effects structure, but be aware this may induce unknown risks of anti-conservative results (i.e., p-values might be lower than nominal).
Can anyone explain me what this means?
Comments
Hi,
This message is a result of a failed convergence check of the likelihood optimizer and indicates that the optimizer might not have converged to the maximum likelihood estimate.
In other words (and greatly simplified), the likelihood is usually an inverse U shaped function. The maximum likelihood estimate can be found by finding parameters resulting in the maximum of the likelihood function. The optimizer tries to find this value by "climbing" up the hill and checking how "steep" the hill is. When you reach the maximum, the "top of the hill", then the slope should be flat (you can climb any higher). This warning message says that when the optimization routine was terminated, the hill was not flat enough, so you might not have reached the maximum.
In the case of a one-dimensional function, the hill is pretty simple to climb, but random effect models can have dozens of parameters (especially with rich random effects structures). The climbing can be very difficult and the highest point unclear.
Unfortunately, there is currently no way to manually tune the optimizers (which is quite a technical thing) in JASP, so the best advice is to try slowly reducing the dimensionality of the model (by removing random effects structure) till the model converges.
(You can find more details regarding the lme4 implementation and optimization routines in this article https://www.jstatsoft.org/article/view/v067i01)
Cheers,
Frantisek
Hi Frantisek,
Thank you for the very good explanation. However, it is unclear to me how to reduce the dimensionality of the model. I have 25 study areas, in each of which I study 10 plants in more detail. I have adjusted the study areas as random effect. Should I try to merge individual study areas? Or is there another way to reduce the random effect structure?
Many greetings
Daniel
Hi Daniel,
You can start removing the individual random effects by deselecting the checkboxes within the "Random effects" specification under the "Model" tab in JASP. First, you should remove the highest order term interaction and then focus on components with the least theoretical relevance or those that have the lowest estimated variance (you can find those by selecting "Variance/correlation estimates" under the "Options" tab).
Cheers,
Frantisek
Hi František,
Thank you very much for your fast answer. I have done as you suggested. Unfortunately, if I add more fixed effects variables to the model, the same error message comes back. This happens even if I have deselected all checkboxes. Are there any other ways to optimize the random effect structure?
Many greetings
Daniel
Hi Daniel,
that's what I would not expect to happen. Could you maybe save the analysis and share the .JASP file with me so I can have a closer look (f.bartos96@gmail.com)?
Cheers,
Frantisek
Hi František,
I will write you an email.
Many greetings,
Daniel
Thanks for sending the analysis file Daniel,
(I'm attaching the reply here as well in case someone else gets into the same problems)
One of your predictors, call it the X variable has relatively large values and they are apparently creating problems in the likelihood evaluation.
I re-scaled the values and the errors disappeared -- the optimizers sometimes have problems with predictors with large values: the regression coefficient gets really small and even a small change in it can create a large difference in the likelihood.
Scaling the predictor does not affect the model, only interpretation of the parameters (you can do it in JASP by clicking a black "+" button in the JASP's data view and adding a new computed column)
When you create a new predictor, e.g., X/10, the new regression coefficient corresponds to 10 times the change in the original X predictor (on the model scale -- in case you use GLMM with a link function, the change is on the scale before applying the link function).
(The t/z statistics and p-values are unaffected by this change since scaling the predictor scales both the coefficient and its standard error.)
Cheers,
Frantisek
Now I have the same problem but I am using a
Binomialfamily and have 3 different fixed effects which are all categories. Now what to do with this warning? :( In R I can add optimizer but in JASP I don't know what I can do.Hi horotat,
Estimating complex GLMMs can indeed be problem. Unfortuntelly, we did not implemented the optimizer control in the JASP interface yet (I just created a feature request: https://github.com/jasp-stats/jasp-issues/issues/1832 so we might address it in the future.) The only other think you could do in JASP right now is trying to simplify the model (especially the random structure), but that might not be the most optimal approach.
Cheers,
Frantisek
Thank you for the feature request @František. So the results of these non-converged analysis are completely not reliable right?
Yes, that's indeed the case.
Frantisek
Hi @František
My question involves understanding the contribution of different random effects to the model fit in order to make an informed decision as to which ones to include.
In order to understand which of my random effects contribute to the model I wish to follow a dual approach (as in Dorman 2019) in which I check both the minimum and maximum contribution of each of the random intercepts. For example, testing the maximum random effect structure (the minimal effect structure will be exactly the opposite), I include all random effects followed by separate steps that decrease in complexity (e.g., first examining the random three-way interaction, then random two-way interactions, then random main effects), involving the removal of random effects not supported by the data. Each step applies multiple iterations and likelihood evaluations to achieve convergence of the final model estimates. At the end of this convergence process, there will be cases when the final model estimates a random effect as one of its boundary constraints, such as exactly zero. This way I can adopt a conservative approach of only removing random effects that explained exactly zero variance.
Currently I can see the while model fit (AIC and LL) when I add or remove random effects but is there any way of seeing the exact contribution of each random factor in JASP?
Since JASP returns the following message (Warning: Numerical problems with the maximum-likelihood estimate (e.g., gradients too large). This may indicate that the specified random effects parameters (random intercepts and random slopes) cannot be estimated from the available data. Consider carefully reducing the random effects structure, but be aware this may induce unknown risks of anti-conservative results (i.e., p-values might be lower than nominal)) I assume it does not fit the optimal random model as a default.
Any help would be appreciated.
Naomi
Hi Naomi,
Currently, you can't see that, unfortunately. You can inspect the random effects variance (which would be an indicator of its contribution) under the "Variance/correlation estimates". If the random effect's variance is essentially zero, it might be a good reason to exclude the component.
By default, JASP fits the maximum random effect structure and then you can reduce it manually.
Hope this helps!
Frantisek