Vignette study 2 X 5 with continuous DV
Sorry if this is a terribly basic question. I have searched and searched for days and asked friends and not been able to come up with a satisfactory answer.
I am working on a vignette study, I’m trying to write the pre-registration but I am getting stuck, I sure would appreciate any help you may have.
Here is a made-up example that has the same properties as my study:
The established theory is that red hammers hurt the very most when whacked into your thumb. They say red hammers hurt more than yellow hammers, blue hammers, and green hammers. I contend that hammers that are made out of metal hurt a lot while hammers made out of nerf don’t hurt hardly at all and that this is totally irrelevant of the color.
Hypothesis: Across different colored hammers there is no difference in the strength of the effect of a metal hammer on individuals’ experience of pain. In other words, there is no interaction of metal hammers (vs. nerf) with hammer color in predicting individuals’ experience of pain.
To test this I will have participants read a vignette that says they got hit by either a metal or nerf hammer and that the color of the hammer was red/green/blue/yellow. I will also have a condition where the participant doesn’t get told what color the hammer is, just whether it is metal or nerf. So I have a 2 (Material: Metal; Nerf) by 5 (Color: Red; Green; Blue; Yellow, No Color listed) design with the DV being how much pain the participant would imagine experiencing.
I thought that I could do a Bayesian ANOVA where I set up material as a two-level factor and color as a five-level factor. I made some test data and tried to do that but it only tells me if there is a difference, and I guess I could do follow-up t-tests with an adjustment of some kind for multiple tests, but a friend said maybe I should use custom contrasts.
When I created the data I made the yellow hammer slightly more painful to make sure that I could find this difference in my test if it is there. And I can see it on the above descriptive plot but I don't know quite how to get to it.
Any suggestions would be GREATLY appreciated! And sorry for the silly example, clearly it has been a long week.
Comments
Also, thanks so much to the team that created JASP; it is delicious!
Well I think you predict:
1. a main effect of material
2. no main effect of color
3. no interaction between material and color
These predictions are straightforward to test with the standard output.
I am not sure what the standard theory says about the effect of material. Presently we do allow post-hoc tests but we haven't implemented contrasts in the Bayesian framework.
E.J.