Contrasts for interactions in repeated measures ANOVA

frederikaust

Hi there,

I'm planning to run a 2-factorial study with a sequential design and I want to analyse it using a Bayesian repeated measures ANOVA in R (the BayesFactor-package). The design is a 3 (A vs. B. vs. C) x 5 (Materials) full crossed design. I have a strong reasons to believe that the outcome for each material will be either A > B > C (if the one hypothesis 1 holds) or A > B = C (if the other hypothesis 2 holds) and I want my test to be as informative as possible to save lab resources. I'm not so much interested in possible variations of the differences (A vs. B or B vs. C) across the 5 materials; I want to know for each of the materials which of the two hypothesis is more likely. My question now is, what is the right way to run this test.

Thanks to Richard Morey's instructive blog posts I know how to specify the equality constraint and I also know how to impose order constraints. Following the example in the blog posts I can easily specify both of my hypotheses for the main effect of my first factor.

My first idea then was to run individual ANOVA for each material, but my intuition is that this might disregard dependencies between the tests and render the Bayes Factors uninterpretable. Is this correct?

My second idea was to fit a model according to hypothesis 2 (A > B = C) including an interaction with the material factor and compare it to 5 more flexible models in which I allow B > C for one of the 5 materials each. Something like this (disregarding the order constraints):

# Define equality constraint
data$equality <- as.character(data$condition)
data$equality[data$equality == "C"] <- "B"
data$equality <- factor(data$equality)

equality_model <- anovaBF(
  old ~ equality * material + id
  , data = data
  , whichRandom = "id"
)

# Relax equality constraint for first material
data$order_1 <- as.character(data$condition)
data$order_1[data$order_1 == "C" & data$material != "1"] <- "B"
data$order_1 <- factor(data$order_1)

order_1_model <- anovaBF(
  old ~ order_1 * material + id
  , data = data
  , whichRandom = "id"
)

c(equality_model, order_1_model)

While this works, I'm uncertain whether this model really does (or even could do) what I hope. When I look at the posterior samples, effects for the deviation of condition B and C are given for all materials (e.g. columns by the name of material:order_1-2.&.B and material:order_1-2.&.C), while these should be the same for all but the first material. So, I assume this way of specifying the contrast introduces a fully crossed factorial model, but because for most materials there is no condition C the model simply recovers the priors. Thus, I suspect that this model specification is consistent with hypothesis 1 (A > B > C) rather than limiting this flexibility to material 1. In consequence, if I understand the statistics correctly this unnecessary flexibility will be punished by the Bayes Factor and will thus bias my analyses against hypothesis 1.

Another option would be to go the other way and compare the most flexible model (A > B > C for all materials) with a slightly more restricted model (A > B = C for one of the materials). This would, however, lead to the same problem except that now I suspect my analyses will favor the more flexible hypothesis 1.

My last idea was to define the constraints for both hypotheses for all materials and compare those models. This way, however, I don't learn anything about individual materials and I suspect this comparison to be less informative (larger expected N in the sequential design) than the more specific comparisons outlined above.

In conclusion, I'm not sure how to test my hypotheses and if this is possible with the BayesFactor package. Any help would be greatly appreciated.

richarddmorey

Hi Frederik, how strongly related are the materials? If you found that an order held for one material, would this lead you to strongly suspect the same ordering for another?

frederikaust

Hi Richard,

sorry for the late response, I missed your reply.

To answer your question: No, it is entirely possible that the flexible model holds for some materials but the restricted model for others. I don't necessarily assume that the outcome for one material is informative for the others.

It is, however, conceivable to categorize the materials in two groups and within each group the relationship you suggest could hold, if this helps.

Cheers,
Frederik