Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

Variability of Bayes Factors

Hi everyone,

I recently read a paper (Pfister, 2021) about the variability of Bayes factors and notably the fact that repeating a Bayesian ANOVA on the same dataset resuts in dramatical changes in Bayes factors. The author of the paper suggested to repeat the analysis a large amount of time and to report the trimmed mean of the Bayes factors. I would like to know whether it is possible to do that in JASP. I saw that there is Repeatability option in the Additionnal Options of the Bayesian ANOVA module where one can specify a seed. However, I am not sure how to use it and I feel that it is not really what I am looking for. I have no doubt that it is possible to use R to deal with this question but I am not familiar enough with R to do so by myself. Can someone help me with that please? Thank you very much in advance.

Best regards

Nico

Comments

  • Hi Nico,

    First, it is generally not a good idea to average BFs. For instance, averaging 1/3 and 3 does not yield 1. So averaging the log BFs would be better. In addition, the variability in ANOVA is quantified by a % error, which you can bring down by increasing the number of samples under Advanced options.

    Indeed, repeatability is not what you are looking for: the results will always be the same (good for reproducibility) but you hide the very variability you wish to reveal and quantify. I am sure you can do this in R (but again, I would average the logs!) Let me email the R master.

    Cheers,

    E.J.

  • Hi Nico,

    I took a look at that paper and noticed that it ignores the error estimate on the BF, which is relevant here. I can't be sure without reproducing their figures, but I suspect the rare outliers in their simulations would be accompanied by large error estimates, so I'd echo what EJ said about paying attention to that, and potentially increasing the number of samples.

    Because all samples are independent, doubling the number of samples is the same as doing it twice and taking the average, with the added benefit that you get an estimate of error as well.

  • edited April 2021

    Hi E.J. and Richard,

    thanks for your answers. When you say I have to increase the number of sample, is that for the posterior samples or for the numerical accuracy option? Where can I get the error of estimate that you mentionned?* Thanks again for your help.

    Best regards,

    Nico

    *Update: Sorry my question was not clear enough on that point. I am working with BFexcl. I found the % error for the regular BF. So my question should have been "Is there a way to get it for BFexcl"? Thanks

  • This is a question for EJ, since BFexcl is his thing. Conceptually, yes, since BFexcl is a function of the individual BFs, but it might be a bit difficult/time consuming to calculate.

  • Thanks Richard. I think it will be convincing enough for my reviewers if I increase the number of samples to decrease the %error. To be sure, the number of samples I have to increase is the one I higlighted in red below, isn't it? thanks again


  • Yes (you can also try it out and see). The BFexcl and BFincl do not come with this error%, but if the individual ones have small error then so will the derivatives.

    E.J.

  • Thanks for your answer EJ and thanks again Richard. Your both helped me a lot.

  • Hi everyone, I was wondering whether averaging the error percentages for all the models included in a given model averaging is useful to have a rough idea of the error percentage that we could get in the BMA? I guess actually computing the error percentages for all BFincl in the BMA requires more complex calculation but I thought that maybe at a conceptual level it could be useful. I would be interested to have your opinion about that. Thank you very much

  • My first thought is that you'd have to weigh the error percentages with the posterior model probabilities, such that error percentages have a bigger impact when they are associated with models that exert a large effect on the end result.

    Cheers,

    E.J.

  • Hi E.J., thanks for your answer. Ok, that makes sense. I will do that.

    Cheers

    Nico

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games