Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Testing for noramlity in JASP

edited November 4 in JASP & BayesFactor

Hi,

I am testing a data set for normality with JASP and I have two questions.

  1. The Kolmogorov-Smirnov test and Liliefors test should (to my knowledge) produce the same D-statistic, but JASP produces two different D-tatistics. Is this a bug in JASP?
  2. The Lillefors test is significant and the other tests are not significant. For the Kolmogorov-Smirnov test this seems logical. For like the Sharpiro-Wilk test this seems not logical, because it is morge powerfull for smaller samplesizes (in this case n=20). What could be the cause fo this?
  3. What could be the best approach/test for testing for normality with n=20?

Thank you in advance.

Begards, Robin

Comments

  • Hi Robin,

    1. The LLMs (who have access the the JASP code, since it is publicly available) suggest that it is a matter of the underlying R calls using a division by n vs. n-1. Can you provide some example data and screenshots? And can you tell us where in JASP you found these tests? (we offer them in several places) I think this issue should go straight to our GitHub page so we can check whether it is a bug (and if so, fix it). See https://jasp-stats.org/2018/03/29/request-feature-report-bug-jasp/
    2. The LLM says: "There is no contradiction here. Higher power means a greater probability of rejection over repeated samples from a specified alternative distribution. It does not mean a smaller \(p\)-value for every individual dataset. (...) The tests summarize departures from normality differently. Lilliefors uses the largest discrepancy between the empirical and fitted normal cumulative distributions. Shapiro–Wilk uses a weighted function of the ordered observations. Shapiro–Wilk performs well against many alternatives, but sample size alone does not establish a universal ranking of tests. The frequently cited comparison by Razali and Wah found Shapiro–Wilk strongest across the alternatives they studied, while also finding limited power at small sample sizes."
    3. Inspect a Q–Q plot and the individual observations

    Look for pronounced skewness, extreme observations, unusually heavy or short tails, clusters, or substantial rounding. A histogram can be particularly unstable with only 20 observations because its appearance depends heavily on the binning. NIST’s guidance explicitly favors normal probability plots over histograms for small samples.

    A Q–Q plot also helps identify

    what kind of departure might be present, which a single \(p\)-value does not tell you. It remains an uncertain diagnostic with \(n=20\); an approximately straight plot cannot establish that the population is normal.

    Check normality for the quantity that matters to the planned analysis

    The relevant assumption depends on the model:

    Intended analysisRelevant normality assumptionPaired \(t\)-testThe population distribution of the

    within-pair differences

    Independent-samples \(t\)-testThe distributions

    within each group

    Normal-error regression or ANOVAThe

    conditional errors

    , assessed through residual diagnostics

    For example, pooling two normally distributed groups with different means can produce a nonnormal pooled distribution without violating the within-group normality assumption. Regression likewise does not require the marginal distribution of the outcome—or of the predictors—to be normal. These distinctions are reflected in the model-diagnostics and normality-pretesting literature.

    Do not interpret nonsignificance as confirmation of normality

    At \(n=20\), failure to reject may simply mean that the sample contains too little information to detect a departure. “The test did not detect nonnormality” is a defensible statement; “the data are normally distributed” is generally too strong.

    Conversely, a significant normality test does not establish that the intended substantive analysis is unusable. The practical effect depends on the type of departure, the design, and the robustness of the analysis. I would therefore avoid using \(p=.05\) from a normality test as an automatic switch between a \(t\)-test and a nonparametric test. The research question and plausible distributions should guide that choice, with a suitable sensitivity analysis where needed. 


    EJ

  • edited November 15

    Hi @Robin46,

    About the D statistic: The KS test in the Distribution module assumes that the parameters are fixed (not estimated from the data) which is why it uses a different denominator. This is also a why the KS test not an appropriate test in most instances. The current setup in the module is suboptimal in that respect and will be fixed in the next version.

    The rest of the points is as @EJ said.

Sign In or Register to comment.