ANOVA post hock power analysis (non bayes)

mcp

Dear all,

I submitted my paper and one of the reviewers ask for the post hock power analysis.

I have 4 independent groups with continuous variable (some of the variable are homoscedastic and some not) so I used Jasp's ANOVA and KRUSAKAL WALLIS to run my analysis with a Tuckey or Dunn Post-hock.

How can I verify the post hock power analysis in Jasp? Do you have any suggestion?

Hope to hear from you soon.. (I have to submit my revision in one week)

EJ

Reporting post hoc power is a terrible idea, as the number is just a transformation of the p-value. You can quote me on that. For a critique see

Hoenig, J. M. and Heisey, D. M. (2001). The abuse of power: The pervasive fallacy of power calculations in data analysis. The American Statistician, 55:19–24

These authors call the approach "fundamentally flawed".

Cheers,

E.J.

mcp

Thank you EJ for your replay,

I know what you meant, but if I really would like to insert that to make the reviewer happy and then cite your critique, how can I calculate that?

EJ

I think you basically do a power analysis using the maximum likelihood estimator as the parameter value under H1, with the same planned sample size you actually had in the experiment. I don't think you can do this in JASP (because I would have put my foot down; we provide p-values but there are boundaries we won't cross).

Cheers,

E.J.

mcp

EJ, you are so helpful,

do you have any resources which I can look at?

like online website, programs, youtube videos, articles etc...

EJ

Hmm this is really not my domain -- I just know of the papers that critiqued the idea. I'd do some Google'ing and feel free to report the results of your search here!

Cheers,

E.J.