agen judi bola , sportbook, casino, togel, number game, singapore, tangkas, basket, slot, poker, dominoqq,
agen bola. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 50.000 ,- bonus cashback hingga 10% , diskon togel hingga 66% bisa bermain di android dan IOS kapanpun dan dimana pun. poker , bandarq , aduq, domino qq ,
dominobet. Semua permainan bisa dimainkan hanya dengan 1 ID. minimal deposit 10.000 ,- bonus turnover 0.5% dan bonus referral 20%. Bonus - bonus yang dihadirkan bisa terbilang cukup tinggi dan memuaskan, anda hanya perlu memasang pada situs yang memberikan bursa pasaran terbaik yaitu
http://45.77.173.118/ Bola168. Situs penyedia segala jenis permainan poker online kini semakin banyak ditemukan di Internet, salah satunya TahunQQ merupakan situs Agen Judi Domino66 Dan
BandarQ Terpercaya yang mampu memberikan banyak provit bagi bettornya. Permainan Yang Di Sediakan Dewi365 Juga sangat banyak Dan menarik dan Peluang untuk memenangkan Taruhan Judi online ini juga sangat mudah . Mainkan Segera Taruhan Sportbook anda bersama
Agen Judi Bola Bersama Dewi365 Kemenangan Anda Berapa pun akan Terbayarkan. Tersedia 9 macam permainan seru yang bisa kamu mainkan hanya di dalam 1 ID saja. Permainan seru yang tersedia seperti Poker, Domino QQ Dan juga
BandarQ Online. Semuanya tersedia lengkap hanya di ABGQQ. Situs ABGQQ sangat mudah dimenangkan, kamu juga akan mendapatkan mega bonus dan setiap pemain berhak mendapatkan cashback mingguan. ABGQQ juga telah diakui sebagai
Bandar Domino Online yang menjamin sistem FAIR PLAY disetiap permainan yang bisa dimainkan dengan deposit minimal hanya Rp.25.000. DEWI365 adalah
Bandar Judi Bola Terpercaya & resmi dan terpercaya di indonesia. Situs judi bola ini menyediakan fasilitas bagi anda untuk dapat bermain memainkan permainan judi bola. Didalam situs ini memiliki berbagai permainan taruhan bola terlengkap seperti Sbobet, yang membuat DEWI365 menjadi situs judi bola terbaik dan terpercaya di Indonesia. Tentunya sebagai situs yang bertugas sebagai
Bandar Poker Online pastinya akan berusaha untuk menjaga semua informasi dan keamanan yang terdapat di POKERQQ13. Kotakqq adalah situs
Judi Poker Online Terpercayayang menyediakan 9 jenis permainan sakong online, dominoqq, domino99, bandarq, bandar ceme, aduq, poker online, bandar poker, balak66, perang baccarat, dan capsa susun. Dengan minimal deposit withdraw 15.000 Anda sudah bisa memainkan semua permaina pkv games di situs kami. Jackpot besar,Win rate tinggi, Fair play, PKV Games
Comments
My understanding is that normally you should look at the Bayes Factor from Study 1 and multiply it by 1/1 if the prior odds of the alternative versus the null hypothesis were 1/1 {50/50}. This yields the posterior odds (which is of course identical to the Bayes factor when the prior odds are 1/1). Subsequently, in Study 2, if you're re-examining the exact same hypotheses (I think many people would reject my 'exactness' requirement), you should take the posterior odds derived from Study 1 and multiply that by the Bayes Factor found in Study 2. The net result is the new, Bayesian-updated posterior odds. (If you're working with BF[10] rather than BF[01], it's the posterior odds of the alternative hypothesis relative to the null hypothesis.)
R
Hi,
thanks for your reply! I'm afraid I need to delve a bit more into understanding the details of Bayesian statistics to know exactly what you mean.
I thought that I could change the default values given in JASP under "Prior" into something that reflects my results from the previous experiment. Does that make sense?
Best, Elena
I believe that the "prior" options in JASP are not options that allow the user to specify prior odds that are different from 1/1. Instead, you should do the following . . .
If (for example) Study 1 yields BF[10] = 2.3, and Study 2 (a replication of Study 1) yields BF[10] = 1.9, then subsequent to the Study 2 analysis, you should compute (by hand or otherwise) 2.3 * 1.9 = 4.37 which equals the "posterior odds." Thus, having gathered and analyzed the data data from the two studies, the likelihood of the alternative hypothesis being true is now 4.37 times the likelihood of the null hypothesis being true.
R
This sounds pretty cool and straightforward. Am I correct to assume that it works just the same with BF[01]s and with more of them? That is, Study 1 yields BF[01] = 3.4, Study 2 yields BF[01] = 1.2, Study 3 yields BF[01] = 3.6, and Study 4 yields BF[01] = 5.1. Then the likelihood of the null hypothesis being true is 74.9 times higher than the likelihood of the alternative hypothesis being true?
However... is 74.9 then the Bayes Factor for the four studies combined? This would confuse me, because if I divide the data set of one of the studies and multiply the two resulting BF[01]s, it does not yield the same value like when I compute the BF[01] for the whole data set. Am I not overestimating the combined BF[01] with the multiplication strategy?
The multiplication works only for point hypotheses, or when the data of the different experiments are independent (in the sense that knowledge about the one experiment does not affect your expectations about the outcome of the other experiment). Let's say you have experiments A and B. Then we have
BF(A,B) = BF(A) * BF(B|A)
The second BF is conditional, that is, it needs to take into account the knowledge obtained from experiment A: this means that the prior distribution needs to be updated. See also https://link.springer.com/article/10.3758/s13428-018-1092-x
Easy example to show that the prior needs to be updated:
We toss a coin. H0: coin is fair (theta = .5); H1: coin is either double-heads or double-tails (theta = 1 or theta = 0). We perform multiple experiments, each consisting of a single toss. Experiment A: the coin lands heads. Now this is equally likely under H0 and H1, so the BF=1. But now consider 1000 successive experiments, each of which will yield "heads". If we multiply without updating the prior distribution, we would conclude that for the complete sequence, BF = 1 (because 1 x 1 x 1.....etc.) Instead, the data are compelling evidence for H1. This comes about because, after Experiment A, the prior distribution for theta under H1 is changed to a point prior on theta = 1 (because the first observation has falsified theta=0); from the second observation onwards, each new observation contributes a factor of 2 in favor of H1. If we have assigned theta a beta distribution, the prior would need to be updated after every observation.
Cheers,
E.J.
Hi,
thanks a lot for this detail. So I guess I shouldn't just multiply BF[01]s, because previous experiments might have already led me to expect data in favour of the H0.
In the first part of this cool paper you cite, you write that the updated distribution [Beta(1 + 20,1 + 10)] can be specified in the Summary Stats module of JASP. And I am sorry that just don't understand how. Does it directly translate into a Cauchy, Normal, or t distribution which I would be able to specify under Priors? [Edit: sorry, I guess I can find that information also elsewhere, eg. https://arxiv.org/abs/1704.02479] Anyhow, as I don't have binomial data (but continuous values), this wouldn't work for me anyway.
From the second part of your cited paper, I understand that I could theoretically just combine all data from my four experiments into one large data set and analyse everything in one step. Is that, so far, correct?
However, there is one more pitfall, as the two conditions were tested between-subject in two of the experiments and within-subject in the other two, I guess I also have to run two separate analyses. I guess it would be highly uncommon to just analyse all four data sets as if all of them were obtained from independent samples for the one combined analysis, right?
And could I even use knowledge from the between-subject experiments for informed priors (however I would enter that information...) for the within-subject experiments? The essential research questions are the same: "are slopes more negative under condition A than condition B?" (where I want to quantify the evidence for the null hypothesis, that slopes are on average just the same under both conditions.)
For continuous priors you have to specify different distributions. I think that in your case I would just select a normal distribution. You can get the best fitting normal distribution from the previous experiment from the mean and 95% central credible interval. Given the complications of design differences this may be more feasible than combining the data.
The effect size delta from the between-subject experiment could be used for the within-subject experiment, I think (but I'll ask our expert to be sure)
From the combined data of the first and second experiment I have the 95% CI [-0.222, 0.488]. Can I directly derive the mean 0.133 (i.e., mean value of the CI)? Then, my first hunch was to simply use the 95% CI as the new normal distribution (something like: a standard deviation for my following prior normal distribution is 1/4 of the range of the 95% CI, thus: mean = 0.133, std = 0,178). Is it really that simply translatable? (If that really works in principle, I could also specify a 95.4% credible interval before calculating the std...)
Furthermore, then the BF is not really meaningful anymore, right? As my prior distribution starts to strongly mass around 0, the BF[01] gets smaller. Would then the only evidence FOR the H0 be the resulting CI?
@EJ
Thanks EJ. I did not know that for typical statistical scenarios--where the alternative hypothesis is a diffuse set of non-zero effect-size possibilities--the statement from the JSP manual, that "Posterior odds Posterior odds = Bayes factor × prior odds" [http://static.jasp-stats.org/Manuals/Bayesian_Guide_v0_12_2_1.pdf], isn't strictly correct since it's an over-simplification.
I do see a conceptual problem, though. "Posterior odds Posterior odds = Bayes factor × prior odds" is a verbal statement of Bayes' theorem! So we are talking about "Bayesian" statistical analyses that do not conform strictly to Bayes' theorem. It might be clearer to describe what we're doing as "quasi-Bayesian" analyses that incorporates only some aspects of Bayesianism.
As an aside, I mentioned my surely-unpopular opinion that prior odds derived from a previous study are only relevant if the current study is an exact replication of the previous study. This is because Bayesian updating (or any kind of updating) of pair of hypothesis requires that the same pair of hypotheses be considered prior to and subsequent to the update. However, if (for example) Study 1 examines the physiological effects of caffeine (from Coca-Cola) on 2nd-year psychology students and Study 2 examines the physiological effects of caffeine (from Pepsi) on 1st-through-3rd year psychology students, Study 1's statistical null hypothesis is different from Study 2's statistical null hypothesis. It is therefore possible for the null hypothesis to be true in one study but false in the other (and the same goes for the alternative hypotheses). For this reason, I think that in real-world *statistical* analysis it's best to always have the prior odds be 1/1, and to "update" by combining all the data into one data set while including Study ID (e.g., 1 vs 2) as an explicit variable. In situations where it seems too difficult or awkward to combine the data across studies, that's probably a good cue that the studies differ in the formulation of their null-hypotheses (and that they differ in the formulation of their alternative hypotheses).
R
I do think the statement "posterior odds = BF times prior odds" is correct; it is just that the calculation of the BF requires some care, that is, you need to update not just your belief in the rival hypotheses but also in the competing parameter values within the hypotheses itself. This is the only way in which the entire system coheres. For examples see https://www.bayesianspectacles.org/bayes-factors-for-those-who-hate-bayes-factors-part-ii-lord-ludicrus-vampire-count-of-incoherence-insists-on-a-dance/ and https://www.bayesianspectacles.org/bayes-factors-for-those-who-hate-bayes-factors-part-iii-the-coherence-plot/
Cheers,
EJ
It's my impression that in Bayes' Theorem, all of the information pertaining to prior beliefs is contained in the 'prior odds' term. I contrast the Bayes factor likelihood ratio is only bout the current data. So I think that any conceptualization of the Bayes factor that reaches back prior to the data at hand is a change to Bayes' theorem rather than a mere application of the theorem.
@esixtus Maybe something to take away from this thread is that the degree of consensus within the practice of Bayesian analyses statistics isn't as strong as the consensus within classical, null-hypothesis testing.
R
There is only a single mathematically correct way to update, if the data are judged to be exchangeable. The second blog post referenced above discusses an example where one analyses a data set of 10 observations in two ways:
(1) all data at once;
(2) in two equally large batches, A and B
The Bayesian inference should be the same in both cases. The only way to achieve this is to do the batch-by-batch analysis by condition on all information, that is, by adjusting the prior distribution.
Of course there is a legitimate debate to be had on whether and to what extent data from prior experiments ought to impact the analysis of the current experiment (maybe there are differences in design or in participant characteristics). If experiments might differ in a meaningful way, a Bayesian random-effects meta-analysis might be the way to go.
Cheers,
EJ
I can see that all is not easy, and maybe not unambiguous, from a philosophical point of view. However, I still wonder, whether in principle my application of EJ's instruction could work (my previous comment, from yesterday, 1.57pm) or whether I fundamentally misinterpreted something...?
Cheers,
Elena
If I may add my two cents:
As far as I understand, you would like to quantify the relative evidence (i.e. Bayes factor) for the effect across all studies, correct?
I, too, would suggest a joint analysis of the raw data. Given that you are combining within- and between-subject studies, a linear mixed model may be appropriate. Currently, JASP supports Bayesian LMM, but does not compute Bayes factors. You could, however, use the R package
BayesFactor, which powers some of JASP's calculations.Here's a simplified example for one within- and one between-subject study. The within-subject experiment has only one observation per participant per condition (otherwise we should add random slopes for the manipulation). Also, I'm making the strong assumption that the true effect size is the same in both studies (otherwise we would need consider additional fixed effects for study or study design).
I'll first simulate some fake data for 100 participants in each study.
n <- 100 subject <- c( 1:n , rep((n + 1):(2 * n), each = 2) ) design <- c( rep("between", n) , rep("within", 2 * n) ) condition <- c( c( rep(0:1, n / 2) , rep(0:1, n) ) ) truth <- data.frame( par = c("mu", "mu_delta", "sigma_mu", "sigma_e") , value = c(0.8, 0.4, 0.75, 0.5) ) m <- rnorm(n * 2, mean = truth[truth$par == "mu", "value"], sd = truth[truth$par == "sigma_mu", "value"]) delta <- rep(truth[truth$par == "mu_delta", "value"], n * 2) dat <- tibble::tibble( subject = subject , condition = condition ) |> dplyr::mutate( dv = rnorm(dplyr::n(), mean = m[subject] + (condition-0.5)*delta[subject], sd = truth[truth$par == "sigma_e", "value"]) , subject = as.factor(subject) , condition = as.factor(condition) , design = as.factor(design) )Now, we can use the
BayesFactorpackage to fit two LMM and compare the relative evidence for the effect. I use very diffuse prior distributions here, which will be inappropriate for your application. Make sure, you understand the nature of the fixed and random effect priors and choose appropriate scales for your effect and design as your results will depend on these settings.If you are new to Bayesian LMMs, our recent special issue on the topic may be of interest. Rouder et al. (2023) provide some guidance on the specification of models and prior scales in their example application (see the section "Fine-Grained Specification", p. 60). They also provide some code for their example analysis. Note that their approach of specifying the null model sparked controversy among the contributors to the special issue. Make sure you understand the implications, consider the alternatives discussed in some of the other contributions, and decide which is most suitable for your application.
Thanks a lot for your response!
I will try that out. It will, however, take quite a while, cause I'm in fact very new to Bayesian LMMs. I was hoping for a "quick and dirty" (but sound) JASP solution with some default priors and just some changed settings. I understand now that this will not work out.
Maybe I'll ask more questions here in a week or in six months...
Cheers
It seems from this thread that all of JASP's Bayesian analyses may be missing crucial bit of output. In my opinion, the most useful aspect of Bayesian (as opposed to a frequentist) analysis is that the former provides a ready answer to the following question: "Given the data, the alternative hypothesis is __ times as likely as the null hypothesis." In other words, the Bayesian analysis provides the posterior odds for the hypotheses.
In a simple, not-uncommon situation where the prior odds are 1:1, a "traditional Bayesian" analysis provides a Bayes factor (rather than posterior odds) as output would present absolutely no difficulty. The posterior odds (OP) would be identical to the Bayes factor (BF) since OP would = BF * 1.0 = BF. However, in light of the preset thread, it seems that such may not be the case when dealing with the kind of statistical Bayesian statistical analysis involving diffuse alternative hypotheses. Therefore, it seems, JASP's (and R's) offering of a Bayes factor doesn't directly provide the most important output of an equal-priors Bayesian analysis: the odds of the alternative versus the null hypothesis, given the data.
R
Agreed in general; some of our analyses (for regression and ANOVA) do present posterior model probabilities though. But perhaps we ought to do this for t-tests as well. I guess we are still prioritizing adding more functionality at the moment...
EJ
Thanks for pointing that out to me. However, below is the JASP output for a Bayesian two-group ANOVA (preceded by a Bayesian t test on the data). I want to know the value of the posterior odds (OP) where: "Given the data, the odds of the alternative hypothesis relative to the null are ______." (*Moreover, in light of the present thread, I expect OP to be something other than just a prior odds of 1.0 multiplied by the present Bayes factor.*) Where in the output do I find that value of OP?
Thanks.
R
Well the posterior odds would be the ratio of the posterior model probabilities (values in the "p(M|data)" column). I personally think that probabilities are a little easier to grasp than odds (although odds are useful for convenient updating, and for working with extreme outcomes).
EJ
Yes, but that ratio appears to always be identical (within the estimation error of the model) to the prior odds of 1:1 multiplied by the Bayes factor, which is simply the Bayes factor (which JASP provides both for the t test output and the ANOVA output). Could this be evidence that with a prior odds of 1:1 it is valid, after all, to compute the posterior odds as an update to the prior odds--i.e., to simply multiply the Bayes factor by 1:1?
R