repeated bayesian ANOVA
I´m new to bayesian statistic and using JASP. So while reading through several articles and playing around with some sample data sets, some questions occurred.
I already conducted a "normal" ANOVA with one between subject factor group and three within subject factors (congruency, affective valence and time). I obtained a significant main effect of affective valence, a signifikant main effect of congruency and two interaction effects congruency * affective valence and time * group.
Now, I want to conduct a bayesian ANOVA, especially because my hypothesis has been that affective valence and group interact with each other, which however was not the case. Thus, I am interested in how much the evidence is for excluding these interaction from the model.
Following questions occurred:
1) With such a four factorial design there are several different models, which I might compare with each other. However I am not sure if it is even necessary (but maybe you`ll tell me otherwise). Basically I am interested how strong the evidence is for including the affective valence * group interaction in the model or rather to exclude it from the model.
So my idea was to:
- first just have a look at which model fits best compared to the null model
- additionally compute the bayesian factor for comparing the model two main effects and interaction effects (affective valence, congruency, affective valence * congruency, time * group) with the model two main effects and three interaction effects (affective valence, congruency, affective valence * congruency, time * group, affective valence*group). Thus, basically comparing a model with all the effects, which were significant in the previous frequentistic ANOVA with a model, which additionally includes the interaction, which was not significant (against my previous hypothesis). So that in the end i for example could say, the data are ..... times more likely under the model without the affective valence * group interaction than under the model including this interaction.
Or would you recommend a different approach?
2) There are some advanced options available in the repeated bayesian ANOVA (r scale fixed effects, r scale random effects and r scale covariates). As I understood from some other posts it is not advisable to change the default values unless you are kind of an expert in bayesian modeling. Still, even though I have no intention to change them, I`d appreciate some explanations on what these values are for. As I understood, the default values have the a priori assumption that each model has an equal probability? Is this correct? If so, in which cases would one assume the models do not hava an equal probability and thus change the default values? Or more specifically, what does each of the default values mean? Why are they set to 0.5, 1 and 0.354 respectively?
3) Is it a valid approach to leave the values as they are and mention in my article that I used the default values or do I have to be afraid that some reviewers will to ask me what I mean by default values and why I did not change them? Don`t want to do anything wrong or over/underestimate the evidence for or agains the different models.
I`d really appreciate any help. Thanks in advance for your time and efforts,