Am I making violations through the use of this test?
It's my first time using JASP as well as Bayesian statistics, so excuse my lack of knowledge. I am trying to figure out whether method A or method B is faster for finding a certain target in a scene. If one of the two methods is indeed faster than the other, I also want to see if this relationship where A is faster than B (or vice-versa) increases as a result of the rarity of target presented (there are 100 possible targets in my data, and each has a different percent chance for showing up in any given scene a person encounters). In other words, figure out whether the methods are different, and whether there is an interaction with target rarity.
There are many limitations in the data I have collected, though. Participants are free to repeat the search task as many times as they want (practice effects are not a concern). Thus, some participants have only a single trial (trial representing a single search for a target in a scene), and others may have hundreds. Participants are also free to use either method A or B as they choose. So some people have data whereby they have only ever used one of the two methods. For those that use both methods, people tend to prefer method B far more often than method A, thus I have different amounts of data for each method. And of course, not every participant has encountered every possible target (as some are very very rare to occur). The one good thing is that I have hundreds of thousands of participants collected for data. Should I be removing participants who have only used one of the two methods, and is it a problem that participants have contributed unequal amounts of data for each method? (Just as an example, participant X may have 20 total trials, 19 of which used method B, and they encountered 10 of the 100 possible targets.) Given all the uncertainty in the data, I was told to try using Bayesian methods.
Can someone point out what analysis I should be using to investigate these issues without falling into traps such as unequal variance and missing data? Is Bayesian Correlation Pairs a statistically sound way to investigate method A vs. method B? And how do I also use JASP to investigate an interaction?
Thanks very much for any help.