Conducting Orthogonal Planned Contrasts within Bayesian ANCOVA in JASP/Bayes Factor
Hi All,
I have a 2 (Variable 1: Choice: Choice and No Choice) x3 (Variable 2: Social modelling: Consistent, Inconsistent and control) between subjects design within an ANCOVA containing one categorical covariate(Gender: Male, Female). I have successfully been able to conduct my Bayesian ANCOVA in JASP and R by dummy coding my covariate.
However, now i would like to get bayes factors for each of my orthogonal contrasts (2 main effects of social modelling, 2 interaction effects between social modelling and choice).
I have tried to contrast code my two variables (Choice and Social Modelling) into 4 variables representing each orthogonal contrast and force them in to the analysis as covariates (Along with gender - dummy coded and choice contrast coded). As a proof of concept i did this within a traditional ANOVA in JASP and the contrast coded 4 orthogonal contrasts did give correct P values.
However, when replicate this process within the bayesian ANCOVA, It does not appear to work (checked by contrasting the BF of the Choice variable compared to the null when entered as a fixed factor in my original bayesian ANCOVA and the BF of the choice variable compared to the null when entered as a contrast coded covariate along with all other variables contrast coded)
I would appreciate anyone that could shed light on why the Bayesian analysis differs from the traditional ANCOVA. Specifically why categorical variables are giving different results when contrast/dummy coded and entered as covariates rather than fixed factors. Is this due to a difference in default priors for fixed factors vs covariates?
Alternatively, if anyone could propose an alternative method though which i could get a BF for an orthogonal contrasts (main and interaction) to report alongside my traditional frequentist statistics.
Thanks
Comments
This is a tricky question. I'll ask Richard and Don to help out.
Cheers,
E.J.
I'm not 100% sure I understand why the "bayesian ANCOVA does not appear to work", a JASP file (or R script) would be very helpful for me to fully understand what you're trying to do. Nevertheless, concerning
Specifically why categorical variables are giving different results when contrast/dummy coded and entered as covariates rather than fixed factors. Is this due to a difference in default priors for fixed factors vs covariates?
That's could definitely be a reason. In JASP, under additional options we have
So the scale of the Cauchy prior is different for variables entered in the box "Fixed Factors" than for dummy coded variables entered in the box "Covariates". If you want to obtain the same results, you should set the "r scale fixed effects" to the same value as "r scale covariates".
Hi Don,
Thanks so much for the response. I have tried changing priors and it does not seem to make a difference.
Here is Data, Output and data dictionary:
(sorry it won't let me upload the JASP file so i have given you my csv and results, if you give me an email i can send you the JASP file)
The issue is when choice (or any categorical variable) is entered dummy coded, the BF01 changes.
e.g. BF choice (category, fixed factor) = 0.310, error 1.4%
BF choice contrast coded, covariate = 0.446, error = 0.01%
Really appreciate your help on this one!
Regards,
Cosette
The priors for continuous covariates are different in more than just the scaling: when you enter variable as a continuous covariates, the prior depends on the covariance matrix. When the variables are truly categorical, the covariance matrix is essentially just contains information about group Ns (This is why we developed Bayesian ANOVA). Additionally, with continuous covariates all covariates are linked via a common g parameter, while this is not the case for categorical variables.
So there are a lot of reasons why the results would be different.
The way to handle this is by recoding the categorical variables. I discuss this a bit on my blog in a simple example (what you can do with recoding: https://bayesfactor.blogspot.com/2015/01/multiple-comparisons-with-bayesfactor-1.html), you'd have to tailor it to your own (somewhat more complicated) needs.
Best, Richard
Thank you Richard for the answer - that clarifies a lot.
I was wondering if you could shed further light on the recoding of the variables. The first contrast (-0.5,-0.5,1) is relatively simple to generate using the guide you linked, however my second contrast compares 2 of three groups specifically (a contrast of -1,1,0 for groups 1 2 and 3 respectively), meaning that when i code the group not of interest as an NA value Jasp/R cannot handle it.
Are there another resources you could suggest, or methods to attempt to recode my 3 level categorical variable into that second 'pairwise' contrast? (preferably in a way that generates the respective interaction as well :) )