How to briefly interpret/report the Bayes factor in papers

Comments

• MSB

  March 2016 edited 3:02AM

  To my understanding, you were correct in your interpretation.
  In order to calculate the odds that the null is correct relative to the alternative, you would have to multiply the BF01 by the prior odds of both hypotheses being correct... (I haven't seen anyone do this in a paper though, is this common practice?)

  M
• EJ

  March 2016 edited 3:02AM

  Hi Senne,

  You were correct, and so is M above. It is the reviewer who has misunderstood.

  Cheers,
  EJ
• senne

  March 2016 edited 3:02AM

  Thank you for your quick and helpful replies!

  Best,

  Senne
• sebastiaan

  March 2016 edited October 2016

  Hi guys,

  While I generally take @EJ's word as gospel on these matters, I'm confused here. My understanding has always been that of the reviewer: The BF is a likelihood ratio that expresses how likely one hypothesis (say H0) is relative to another hypothesis (say H1) given a set of data. This is also how the Bayes factor is described in this article:

  • http://www.psychologicalscience.org/index.php/publications/observer/2015/september-15/bayes-for-beginners-probability-and-likelihood.html

  Which says:

  To decide which of two hypotheses is more likely given an experimental result, we consider the ratios of their likelihoods. This ratio, the relative likelihood ratio, is called the “Bayes Factor.” œ

  Reading the discussion above, it's almost like the following statements are equivalent (or at least confused) if we assume that H0 and H1 are equally likely to begin with:

  1. BF01 expresses the likelihood of H0 relative to H1 given the data.
  2. BF01 expresses the probability of the data given H0, relative to H1.

  Are these two statements equivalent? If so, then at least option 1 is the more common interpretation, right? That's what the reviewer means; that's what's in the Psych Sci article; and that's also my understanding. But option 2 seems to be what @MSB and @EJ mean.

  My apologies if my comment only adds to the confusion. But I think this warrants a few extra words.

  Cheers,
  Sebastiaan

  
  
  Check out SigmundAI.eu for our OpenSesame AI assistant!
  
• EJ

  March 2016 edited 3:02AM

  Hi Sebastiaan,

  Statements 1 and 2 are not identical. They can be numerically the same but only if the models are equally plausible a priori. Consider for instance ESP. In a given experiment, the data may be 10 times more likely under H1 (there is ESP) than under H0 (there is no ESP). In other words, H1 predicted the data from the experiment better than H0. This is the Bayes factor: the relative plausibility of the data under H1 versus H0. But this does not mean that we can conclude that it is 10 times more likely that people have ESP! The a priori probability of ESP is very very low, so a posteriori (combining the prior odds with the BF) the plausibility of ESP is still low, even though the experiment provided some evidence in its favor.

  Cheers,
  E.J.
• sebastiaan

  March 2016 edited 3:02AM

  Right, so then my statement was correct?

  Reading the discussion above, it's almost like the following statements are equivalent (or at least confused) if we assume that H0 and H1 are equally likely to begin with:

  Or, phrased differently, you could say: BF01 is a likelihood ratio that reflects the likelihood of H0 compared to H1 given a set of data, but given only this set of data and not taking into account any other data that might affect the likelihood of the hypotheses.

  So the reviewer is not really incorrect, then. He's just interpreting the BF how most people would interpret it, implicitly assuming that H0 and H1 are equally likely to begin with.

  
  
  Check out SigmundAI.eu for our OpenSesame AI assistant!
  
• EJ

  March 2016 edited 3:02AM

  Well yes, the BF equals the posterior odds under equal prior odds. But the reviewer was not saying this or implying it; from what I understood, the reviewer was trying to convince the authors that the theoretically correct interpretation of the BF is in terms of posterior odds, not as an average likelihood ratio. So the reviewer was saying that the BF is about p(H|data) instead of p(data|H), whereas it is exactly the other way around.
  Cheers,
  E.J.
• sebastiaan

  March 2016 edited 3:02AM

  So the reviewer was saying that the BF is about p(H|data) instead of p(data|H), whereas it is exactly the other way around.

  Fair enough, that's indeed what he says. But so does Gallistel in his Psych Sci column. Would you say that he makes the same mistake of confusing p(H|data) with p(data|H)?

  
  
  Check out SigmundAI.eu for our OpenSesame AI assistant!
  
• EJ

  March 2016 edited 3:02AM

  Gallistel writes "To decide which of two hypotheses is more likely given an experimental result, we consider the ratios of their likelihoods." The issue is how much emphasis to place on "given an experimental result". If you interpret this as "the evidence that is in the data" or "the degree to which the data change our beliefs" then he is correct. If he means it defines p(H0|data)/p(H1|data), then he is incorrect.
• FSella

  July 2017

  Hi,

  I jump in the discussion maybe a little bit late, but I am actually facing the same issue. I ran some analyses using JASP (BTW, thanks to the team for the amazing software) and reported BF10 in the paper. Now, I would like the reviewers (and myself ) to correctly get the meaning of BF.

  That's what I wrote at the beginning of the Results section:

  "Statistical analyses were conducted using the free software JASP using default priors (JASP Team, 2017). We reported Bayes factors expressing the probability of the data given H1 relative to H0 (i.e., values larger than 1 are in favour of H1) assuming that H0 and H1 are equally likely".

  Does it sound right?

  Thanks in advance for any help.

  Best wishes

  Francesco
• EJ

  July 2017

  Hi Francesco,

  You are reporting the BF, which, as you indicate, express the relative likelihood of the data under the models at hand. So that means that you can eliminate the text "assuming that H0 and H1 are equally likely" -- the analysis does not assume that.

  Cheers,
  E.J.
• FSella

  July 2017

  Many Thanks E.J.!