Testing for noramlity in JASP
Hi,
I am testing a data set for normality with JASP and I have two questions.
- The Kolmogorov-Smirnov test and Liliefors test should (to my knowledge) produce the same D-statistic, but JASP produces two different D-tatistics. Is this a bug in JASP?
- The Lillefors test is significant and the other tests are not significant. For the Kolmogorov-Smirnov test this seems logical. For like the Sharpiro-Wilk test this seems not logical, because it is morge powerfull for smaller samplesizes (in this case n=20). What could be the cause fo this?
- What could be the best approach/test for testing for normality with n=20?
Thank you in advance.
Begards, Robin
Comments
Hi Robin,
Look for pronounced skewness, extreme observations, unusually heavy or short tails, clusters, or substantial rounding. A histogram can be particularly unstable with only 20 observations because its appearance depends heavily on the binning. NIST’s guidance explicitly favors normal probability plots over histograms for small samples.
A Q–Q plot also helps identify
what kind of departure might be present, which a single \(p\)-value does not tell you. It remains an uncertain diagnostic with \(n=20\); an approximately straight plot cannot establish that the population is normal.
Check normality for the quantity that matters to the planned analysis
The relevant assumption depends on the model:
Intended analysisRelevant normality assumptionPaired \(t\)-testThe population distribution of the
within-pair differences
Independent-samples \(t\)-testThe distributions
within each group
Normal-error regression or ANOVAThe
conditional errors
, assessed through residual diagnostics
For example, pooling two normally distributed groups with different means can produce a nonnormal pooled distribution without violating the within-group normality assumption. Regression likewise does not require the marginal distribution of the outcome—or of the predictors—to be normal. These distinctions are reflected in the model-diagnostics and normality-pretesting literature.
Do not interpret nonsignificance as confirmation of normality
At \(n=20\), failure to reject may simply mean that the sample contains too little information to detect a departure. “The test did not detect nonnormality” is a defensible statement; “the data are normally distributed” is generally too strong.
Conversely, a significant normality test does not establish that the intended substantive analysis is unusable. The practical effect depends on the type of departure, the design, and the robustness of the analysis. I would therefore avoid using \(p=.05\) from a normality test as an automatic switch between a \(t\)-test and a nonparametric test. The research question and plausible distributions should guide that choice, with a suitable sensitivity analysis where needed.
EJ
Hi @Robin46,
About the D statistic: The KS test in the Distribution module assumes that the parameters are fixed (not estimated from the data) which is why it uses a different denominator. This is also a why the KS test not an appropriate test in most instances. The current setup in the module is suboptimal in that respect and will be fixed in the next version.
The rest of the points is as @EJ said.