How to conduct sensitivity analysis in one-way ANOVA?

deirdreoshea

Hi!

 I am trying to write up a paper using Bayesian analysis conducted in JASP. It is for an experimental design where the DV is the log-transformed change score in theta band activity at an EEG channel and the IV the experimental condition (with 3 conditions).

I’ve run the one way Bayes ANOVA and I think I understand most of it. However, I am getting a bit stuck on the prior – I have just used the default prior based on Wagemakers et al (2018) discussion of objective vs subjective approaches and also Morey's 2012 paper on ‘default Bayes factors for ANOVA design’.

However van der Schoot recommends that when reporting Bayes analysis one should include a sensitivity analysis by comparing the findings with different values of priors – and this is where my (lack of) understanding brings me to a halt! I followed the fact that the default priors are based on a Cauchy distribution but to be honest I have no idea what the actual ‘number’ in the default prior in JASP means or represents, nor what I should change it to for this sensitivity analysis. I guess I should input values that narrow or widen the prior distribution but I don’t know what these numbers should be nor why!?

Thanks in advance for your help!

Deirdre O'Shea

EJ

Hi Deirdre,

Yes this is tricky. The ANOVA is specified through differences from a grand mean. Resources & ideas:

1. This paper: http://www.ejwagenmakers.com/2017/RouderEtAl2017ANOVAPM.pdf (Rouder, J. N., Morey, R. D., Verhagen, A. J., Swagman, A. R., & Wagenmakers, E.-J. (2017). Bayesian analysis of factorial designs. Psychological Methods, 22, 304-321.)
2. You could eliminate one condition and see how the results map on to the t-test. I believe that under the default specification, the results should match (but the prior width is off by a factor of 1/2, because of how things were coded).
3. In general, you could do post-hoc t-tests for the individual comparisons, and do robustness checks for them. I find it easier to do robustness checks when the underlying model is simpler.
4. JASP guidelines for reporting a statistical analysis: https://psyarxiv.com/yqxfr

Cheers,

E.J.

deirdreoshea

Thank you very much EJ! I'll read those papers and hopefully that will have me sorted!

Best wishes,

Deirdre