Sampling Plan for Bayesian Contingency Tables
I am having a hard time distinguishing a choice in the sampling plan between the joint multinomial and independent multinomial. I have read other posts on this forum as well as the Jamil 2016 article, but still confused. Here is how I sampled. I collected data for 200 subjects and then stopped. Then assigned to one of two conditions (control and experimental) where each condition is randomly assigned 100 subjects.
On the one hand, I can see this as joint multinomial because n is fixed (200), and the assignment is random--I did not select subjects to be in either group based on any criteria. Per Jamil et al., "...this scheme holds when the stopping rule is 'collect data from 100 cars and then stop.'
On the other hand, it seems like a standard experimental design that people have been advised to use the independent multinomial because the comparing the groups which are fixed (but randomly assigned). Jamil et al., state, "...assume sampling based on independent multinomial scheme, such that the crucial test involves a comparison of two proportions." And in the example in the paper, they say subjects were picked based on sex (males = 50 and females = 50); then again the Dutton and Aron example at the beginning of the paper is (presumably) includes a random assignment.
The DV is in the rows and the IV is in the columns.
I suppose it is the random assignment that is throwing me off. Any advice would help.
Comments
Dear Radamanthus,
First, I would strongly suggest the Bayesian A/B test for this design; see https://psyarxiv.com/z64th and https://onlinelibrary.wiley.com/doi/10.1002/sim.9278 for instance.
Second, this is an independent multinomial. The joint multinomial means that you have four cells, and that counts can fall in either one (for instance, all counts can fall into a single cell) -- this is not the case for your design.
Cheers,
E.J.
EJ,
Thank you for your response. That makes sense about the independent vs. joint multinomial choice. It might help to have some clear examples in the help file when updated in JASP.
I initially ran the Bayesian A/B test in JASP, but I was concerned that reviewers would be less receptive and thought the Bayesian contingency analysis would provide a more familiar analysis frame to incorporate the BF and posterior logged odds. But having read the Dablander et al., 2022 paper you suggested above, I see why the recommendation for Bayesian A/B testing over contingency tables.
Indeed, the A/B provides a higher BF10 than the contingency table in my analysis. Thank you so much for the reference!