Which test to use?
I work in metabolomics research, and one particular statistical method that I use involves the following process:
- Conduct a univariate test (e.g., independent samples t-test) on each chemical compound, normally 500 or more
- Group the chemical compounds based on various factors such as structure
- For each group of compounds, use a Kolmogorov-Smirnov test with a uniform distribution to see if there is a sufficient positive skew to examine effects at the compound group level (i.e., are there more compounds with p values less than 0.05 than might be expected if there was no effect on compound group?)
I'm wondering if some Bayesian version of this might exist, where BFs are calculated for each individual compound (which is simple enough through t-tests), then some way of agglomerating these values based on the chemical group to get a chemical group-level BF?