Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Supported by

How to briefly interpret/report the Bayes factor in papers

edited March 2016 in JASP & BayesFactor

We recently used JASP to calculate the Bayes factor for a couple of experiments in a study that is now under revision. Thank you for this very helpful and intuitive software!

One of our Reviewers now suggests that we misinterpreted the Bayes factor. To be honest, we are a bit confused/uncertain on how to reply. Although the comment does make sense to us, we did inspire our phrasing on an earlier paper using the JASP Bayes factor and were wondering if we could consult your expertise on this?

Specifically, we mentioned:
"The BF01 was 11.499, suggesting that these data are 11.499 more likely to be observed under the null hypothesis."

The reviewer commented:
"According to my understanding, the Bayes factors tell the relative odds that the (in this case) null hypothesis is correct relative to the alternative hypothesis, given these data. Namely, it is not the probability of the data (given the hypothesis) which is what null hypothesis testing tell us, but the probability of the hypothesis given the data."

Comments

  • MSBMSB
    edited 8:56PM

    To my understanding, you were correct in your interpretation.
    In order to calculate the odds that the null is correct relative to the alternative, you would have to multiply the BF01 by the prior odds of both hypotheses being correct... (I haven't seen anyone do this in a paper though, is this common practice?)

    M

  • EJEJ
    edited 8:56PM

    Hi Senne,

    You were correct, and so is M above. It is the reviewer who has misunderstood.

    Cheers,
    EJ

  • edited 8:56PM

    Thank you for your quick and helpful replies!

    Best,

    Senne

  • edited October 2016

    Hi guys,

    While I generally take @EJ's word as gospel on these matters, I'm confused here. My understanding has always been that of the reviewer: The BF is a likelihood ratio that expresses how likely one hypothesis (say H0) is relative to another hypothesis (say H1) given a set of data. This is also how the Bayes factor is described in this article:

    Which says:

    To decide which of two hypotheses is more likely given an experimental result, we consider the ratios of their likelihoods. This ratio, the relative likelihood ratio, is called the “Bayes Factor.” œ

    Reading the discussion above, it's almost like the following statements are equivalent (or at least confused) if we assume that H0 and H1 are equally likely to begin with:

    1. BF01 expresses the likelihood of H0 relative to H1 given the data.
    2. BF01 expresses the probability of the data given H0, relative to H1.

    Are these two statements equivalent? If so, then at least option 1 is the more common interpretation, right? That's what the reviewer means; that's what's in the Psych Sci article; and that's also my understanding. But option 2 seems to be what @MSB and @EJ mean.

    My apologies if my comment only adds to the confusion. But I think this warrants a few extra words.

    Cheers,
    Sebastiaan

  • EJEJ
    edited 8:56PM

    Hi Sebastiaan,

    Statements 1 and 2 are not identical. They can be numerically the same but only if the models are equally plausible a priori. Consider for instance ESP. In a given experiment, the data may be 10 times more likely under H1 (there is ESP) than under H0 (there is no ESP). In other words, H1 predicted the data from the experiment better than H0. This is the Bayes factor: the relative plausibility of the data under H1 versus H0. But this does not mean that we can conclude that it is 10 times more likely that people have ESP! The a priori probability of ESP is very very low, so a posteriori (combining the prior odds with the BF) the plausibility of ESP is still low, even though the experiment provided some evidence in its favor.

    Cheers,
    E.J.

  • edited 8:56PM

    Right, so then my statement was correct?

    Reading the discussion above, it's almost like the following statements are equivalent (or at least confused) if we assume that H0 and H1 are equally likely to begin with:

    Or, phrased differently, you could say: BF01 is a likelihood ratio that reflects the likelihood of H0 compared to H1 given a set of data, but given only this set of data and not taking into account any other data that might affect the likelihood of the hypotheses.

    So the reviewer is not really incorrect, then. He's just interpreting the BF how most people would interpret it, implicitly assuming that H0 and H1 are equally likely to begin with.

  • EJEJ
    edited 8:56PM

    Well yes, the BF equals the posterior odds under equal prior odds. But the reviewer was not saying this or implying it; from what I understood, the reviewer was trying to convince the authors that the theoretically correct interpretation of the BF is in terms of posterior odds, not as an average likelihood ratio. So the reviewer was saying that the BF is about p(H|data) instead of p(data|H), whereas it is exactly the other way around.
    Cheers,
    E.J.

  • edited 8:56PM

    So the reviewer was saying that the BF is about p(H|data) instead of p(data|H), whereas it is exactly the other way around.

    Fair enough, that's indeed what he says. But so does Gallistel in his Psych Sci column. Would you say that he makes the same mistake of confusing p(H|data) with p(data|H)?

  • EJEJ
    edited 8:56PM

    Gallistel writes "To decide which of two hypotheses is more likely given an experimental result, we consider the ratios of their likelihoods." The issue is how much emphasis to place on "given an experimental result". If you interpret this as "the evidence that is in the data" or "the degree to which the data change our beliefs" then he is correct. If he means it defines p(H0|data)/p(H1|data), then he is incorrect.

  • Hi,

    I jump in the discussion maybe a little bit late, but I am actually facing the same issue. I ran some analyses using JASP (BTW, thanks to the team for the amazing software) and reported BF10 in the paper. Now, I would like the reviewers (and myself :smile:) to correctly get the meaning of BF.

    That's what I wrote at the beginning of the Results section:

    "Statistical analyses were conducted using the free software JASP using default priors (JASP Team, 2017). We reported Bayes factors expressing the probability of the data given H1 relative to H0 (i.e., values larger than 1 are in favour of H1) assuming that H0 and H1 are equally likely".

    Does it sound right?

    Thanks in advance for any help.

    Best wishes

    Francesco

  • Hi Francesco,

    You are reporting the BF, which, as you indicate, express the relative likelihood of the data under the models at hand. So that means that you can eliminate the text "assuming that H0 and H1 are equally likely" -- the analysis does not assume that.

    Cheers,
    E.J.

  • Many Thanks E.J.!

Sign In or Register to comment.

agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq, agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq , dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games