Correction for multiple tests (a lot of...)
I will start by saying that I am novice in Bayesian statistics, so I am really sorry if I mixed up some notion. I’m currently doing a research on the effect of heat on some bone microstructure. I want to test if there is any difference on the size of structure when exposed to heat. I have 45 bone, and each of these 45 bones has N=120 non-heated microstructure and N=120 heated microstructure, approximately. I wanted to do a Bayesian Mann-Whitney U test (in JASP) for non-heated structure vs heated structure of my 45 bones and for 5 variables (area, diameter, circularity, etc.), but I’m concerned about the multiplicity of tests and how to handle it.
In what I’ve seen so far, a correction can be applied by changing prior odds and then multiplying those with BF10 to get corrected posterior odd. My first question is: How important are the posterior odds in the interpretation of a Bayesian Mann-Whitney U test, since they are not reported in JASP for individual test (only with a post-hoc ANOVA) and for what I understand, BF are not concerned with the correction. With that many tests, I will certainly get a very important correction that will favor the null hypothesis. Are they any limits to the correction?
I choose to test each bone separately because bone microstructure is really variable across individuals and bone type. My goal by doing so many tests is to see if there is a global tendency and I’m worried that I would lose all evidence of an effect if I applied a correction. My argument for not doing a correction is : with that many tests, even if I have some false positive, my goal is to see the global tendency and those would not matter as much as if I would rely on each individual test alone to answer my question. Does that make any sense? I’m kind of lost here and any help would be very much appreciated.
Thanks a lot