Different results on each test running
Hi everyone!
I'm using JASP for bayesian inference, but I got a problem:
whe I run a bayesian independent testing(Man-Whitney) I get some results, but if I open the same database and run again the testing I got different results (Bayesian factor)
also, if when I open a SPSS database, if perform directly the tests on that database I got some results, but If I convert that database to jasp format and I run the test, results change
If I run the test on many vars at the same time I got a result, but I run the same test just a few vars the bayes factor changes!
why? which outcome is credible?
Thanks in advance!!!
ps: I'm using last version of JASP, default prior and same parameters on every run
Example:
Result 1 database in jasp format
Same database, same data and vars, computed with less vars selected:
Same database and vars, but this time the testing was performed on the spss format database:
Help!!!
Comments
Well I'm pretty sure that is no meaningful difference. As far as I understand this the variation for BF around 1 can be that high by design. If BF is near to 1 you can't draw any inference about H0 or H1 being more adequate. This variation probably comes from the different pathes the MCMC chains took and the generated posterior samples they produced which are pretty close but not identically. I hope this information helps you further.
You may have a look at this table about BF size interpretation as a mere guideline for your interpretation: Felix Schönbrodt's blog (nicebread.de)
Well I'm pretty sure that is no meaningful difference. As far as I understand this the variation for BF around 1 can be that high by design. If BF is near to 1 you can't draw any inference about H0 or H1 being more adequate. This variation probably comes from the different pathes the MCMC chains took and the generated posterior samples they produced which are pretty close but not identically. I hope this information helps you further.
You may have a look at this table about BF size interpretation as a mere guideline for your interpretation: Felix Schönbrodt's blog (nicebread.de)
Indeed, it is as Famondir says. The differences are due to the stochastic nature of the MCMC process. By increasing the number of MCMC iterations this variability can be reduced, but it will never be zero exactly.
E.J.
Thank you for the reply and sorry for the delay,
you are right, Now i know that's specified in the JASP bayesian guide
:)