Use of Bayesian ANCOVA for PhD Thesis - help to ensure correct interpretation of data
Hello Team,
I am new to Bayesian statistics but I am already hooked and have thoroughly enjoyed learning and applying this approach to my research questions in my PhD Thesis. However, I am lacking a bit of confidence in my interpretation of the output and was hoping that you could help confirm my interpretation is correct, and help guide me to the correct interpretation if and where I have not interpreted it correctly.
I am interested in looking at the effects of presence and action on grief symptom severity in relatives who experience the sudden death of a loved one following an out-of-hospital cardiac arrest.
I have a fixed factor of Group with 3 levels based on presence and action at the time of the event:
Group 1 - Witnessed the event and attempt CPR (WCPR)
Group 2 - Witnessed the event and did not attempt CPR (WnoCPR)
Group 3 - Did not witness the event/unwitnessed (UW)
My DV is a continuous measure of symptoms of complicated grief.
As I am also interested in changes in symptom severity over time (Time 1: 3-6 months after the event; Time 2: 6-9 months after the event), I decided to first run a mixed factorial ANOVA. There was a significant main effect of Time. There was no significant effect of Group, but there was a significant interaction. The interaction effect indicates a significant increase in symptom severity over time for those who witness the event and fail to attempt CPR (Group 2) and those who do not witness the event (Group 3), but a reduction in symptom severity over time in those who witnessed the event and attempted CPR (Group 1).
Due to small sample size and unequal group sizes, I wanted to use Bayes to help provide additional context and understanding of what is important in terms of predicting a poor outcome following this event.
Below is the output for a mixed Bayesian ANOVA. I would be extremely grateful if you could check my interpretation is correct.
"Additional analysis of the effects of time and presence and action on severity of grief reactions was carried out using a Bayesian Factorial ANOVA. This revealed that the observed data are 16.46 (BF10) times more likely under the model that includes the main effect of Time, the main effect of Group, and an interaction between Time and Group as predictors than under the null model. Evidence strongly suggests that the observed data are 15.93 times more likely to occur under a model which contains an interaction between Time and Group. The observed data is not sufficiently informative to allow a strong conclusion about the effect of Group (BFinclusion = 1.25), however the addition of Time weakens the model (BFinclusion = 0.83)."
Given the high rates of comorbidity identified in this population, the effect of presence and action on severity of grief reactions following a failed resuscitation attempt of a loved one were explored further using a between subjects ANCOVA, controlling for the potential confounding influence of comorbidities of PTSD, anxiety, and depression as covariates. Symptoms of PTSD violated the assumption of independence and was therefore excluded as a covariate from the analysis. All other assumptions were met. At Time 2, there was a significant effect of the covariate’s anxiety [F (1,51) = 10.536, p = .002, ŋ² = .171] and depression [F (1,51) = 5.217, p = .027, ŋ² = .093] on severity of grief reactions. Whilst controlling for anxiety and depression, a significant effect of presence and action (group) on grief symptom severity was found [F = 4.417, p = .017, ŋ² = .148], representing a large effect size. Those who witnessed the event and failed to attempt CPR (Group 2) had statistically greater symptom severity for CG than those who witnessed the event and did attempt CPR (p = .021), indicating a medium effect size (d = -.591). There was no significant difference in CG symptom severity between those who witnessed the event and those who did not. Having applied the adjustment for the covariant, marginal means indicate that taking into account comorbid anxiety symptoms, those who witness the event and fail to attempt CPR have a slight reduction in grief symptom severity, whereas there is a slight increase in grief symptom severity in those who witness the event and attempt CPR and those who do not witness the event.
Again, I would be hugely grateful if you could check the interpretation of the Bayesian ANCOVA output below:
"We also performed a Bayesian analysis of covariance on severity of grief reactions at Time 2, including Group based on presence and action (WCPR, WnoCPR, UW) as a fixed factor and anxiety and depression symptoms at Time 2 as a covariate. The Bayesian ANCOVA works by comparing eight models with varying predictors of grief symptom severity at Time 2: (1) a null model; (2) a model containing only Group as a predictor; (3) a model containing only anxiety as a predictor; (4) a group containing only depression as a predictor; (5) a model containing both anxiety and depression as predictors; (6) a model containing Group and anxiety as predictors; (7) a model containing Group and depression as predictors; and (8) a model containing Group, anxiety and depression as predictors. Only models 6 and 8 had their model odds increased after observing data (BFM = 2.55 and BFM = 7.85, respectively). Of these, model 8 was the most probable, P(M/data) = 0.529, and the observed data was 1.98 more likely under model 8 than model 6.
To account for model uncertainty, we performed Bayesian model averaging to test the “effect” of all predictors. The data were 39.02 times more likely under models containing anxiety as a predictor, 4.47 times more likely under models including Group as a predictor, and only 2.02 times more likely under models containing depression as a predictor. Thus, we believe that grief symptom severity at Time 2 is affected by one’s presence and action at the time of the event, with those who witness the event and fail to attempt CPR having on average a 4.40 increase in symptom severity, but critically, that the strength of this effect is strongly impacted by comorbid symptoms of anxiety and depression".
Thank you for any advice you can give!!
Comments
I'll comment on the first analysis: this is fine, except for the formulation "Evidence strongly suggests that the observed data are 15.93 times more likely to occur under a model which contains an interaction between Time and Group." It is a *fact* that, given your models, "the observed data are 15.93 times more likely to occur under a model which contains an interaction between Time and Group". This can be categorized as strong evidence. But it is not correct to say that there is strong evidence that the data are 15.93 times more likely. I'd also say that the comparison between the full model and the two main effects model is the critical comparison here.