ANOVA analysis on non-normal distributed data

Coelen

I've got a dataset with roughly 20k datapoints. The distribution is not normal. To what extend is it advised to do bayesian ANOVA?

EJ

I would be highly surprised if --with 20k datapoints-- the non-normality affects the qualitative conclusions. What you can do to check is transform the data to (approximate) normality and carry out the test on the transformed data. With 20k participants, one would expect similar results. (although note the impact of monotonic transformations on the interpretation of interactions, see http://www.ejwagenmakers.com/2012/WagenmakersEtAl2012Loftus.pdf).

Coelen

Thanks for your response. I do not have a lot of experience with transforming data, so for now I will run with the non-normal data. I have some trouble with interpreting the results attached. The BayesFactors are very large. If I remember your dartboard analogy from the workshop, this leads me to believe that this shows strong evidence for 'maincat' and 'group'. However, when looking at P(M|Data), the null hypothesis seems very unlikely. How would you interpret these numbers?

Some background on the experiment: We measured a specific conversion rate on an online platform. We had 3 groups of participants: a control group, and two groups who received a different treatment. We tested this conversion rate over 3 categories (maincat).

EJ

Sorry for the tardy response! Do you mean that "group" seems unlikely, even though there is a lot of evidence to prefer it over the null? This is because the BF_10 column compares each model against the null, whereas the P(M|data) column takes into account all models. This is why I think the "effects" table tells the story: you absolutely need the Maincat variable, and it is so dominant that it overshadows any effect of Group -- the evidence for including that variable is inconclusive. I'm working on a tutorial paper that explains the output in more detail! Hope to be done in a few weeks. I'll post this on the Twitter account when it's finished.