Bayesian Mixed ANOVA and power analysis

Michif

Hi,

I searched this forum in the hopes for an answer to the following question:

How do I calculate a-priori power, or my required sample size to obtain a certain effect size, for Bayesian ANOVAs, particularly for a mixed ANOVA where the effect of interest is the interaction term between the between- and the within-subject factor?

I found this app (https://www.bayesianspectacles.org/an-interactive-app-for-designing-informative-experiments/) which only does t-tests, and this one-year old conversation (https://forum.cogsci.nl/discussion/5478/bayesian-power-analysis-for-calculating-sample-size-with-fixed-n) with regards to the alternative approach that suggests using G*Power (i.e. the frequentist approach) while reducing the alpha to .005.

I haven't found anything in JASP so I assume the power analysis functionality has still not been implemented in JASP? Does then the same suggested alternative approach also hold for my specific problem (i.e. reduce the alpha)?

Additionally, say if I wanna answer the question of whether there is evidence for the H0 instead of H1 in this specific case, would I still calculate the power in the same way (i.e. treat it as if I am looking for evidence in favor of the H1)?

Any help is appreciated - thank you!

EJ

Hi Michif,

Sorry for the tardy response.

1. Yes, the app you found only does t-tests
2. Correct, the power analysis functionality is still missing in JASP
3. Reducing alpha to .005 is indeed a sensible alternative approach
4. The classical concept of power refers to the probability of detecting a true non-zero effect. If you want to "power" your experiment to have reasonably strong evidence in favor of H0, you will need more observations than when you are powering for a non-zero effect -- under the usual assumptions, it is just easier to detect an effect that is present than it is to conclude that an effect is absent.
5. We ought to do more work on this, it is clearly important.

Cheers, E.J.

Michif

Thanks Eric-Jan for the clarification.

You guys are doing a really great and valuable job with JASP. Please keep it up!

rohanp16

Seconded regarding a power analysis function for Bayesian mixed ANOVAs :)