I teach postgrad psychology stats and I am wanting to include JASP and Bayes beyond a basic level in the course. The programme director is a big fan of MM/SEM and bootstrapping, s/he seems to think that this in combination with bootstrapping is the ONLY way forward in the replication crisis. I‘ve struggled to find papers or textbooks supporting this and I‘d quite like my students to learn Bayesian modeling...is there a way of running th above analysis in JASP?

Thank you so much,

eniseg2

]]>I can't seem to find any trace plots, or indication how many iterations were needed, so I am uncertain how to claim convergence other than trusting JASP to tell me if anything was wrong, and redoing the analysis a couple of times to check if results don't vary too much. ]]>

From what I understand, one can set the Cauchy prior width acordding to a speculated effect size when preforming a Bayesian t-test, such that if I think my effect should be around 0.6, I would set the Cauchy prior width to 0.6 (correct me if I'm wrong).

What I still don't understand is the Beta* prior width for correlations - can it be used in the same manner?

If I expect an r=~0.6, would I set the Beta* prior width to 0.6?

Or in other words, how do I translate an expected effect size/correlation to Beta* prior width?

Thanks,

Mattan

I would also love if anyone has published examples of how to appropriately write a results section using Bayesian ANOVA?

Thanks in advance

Andrew ]]>

Since neither JASP nor the BayesFactor package include robust tests, I am considering to compute Yuen's robust t-test and robust multiple regression through a M-estimator and then enter the data of t and R-squared in the "summary stats" of JASP to compute the Bayes factors of the robust tests. Does this procedure makes sense?

]]>Whilst running a normal contingency table and ticking Phi and Cramers V - I see that that Cramers V (probably the most robust) is calculated and Phi reports NaN - Value could not be calculated - at least one row or column contains all zeros

Data sheet looks fine to me.

Any ideas?

Mark

After reading about the ANOVA model implemented in the BayesFactor package (Rouder, Morey, Speckman, & Province, 2012) a while ago I wondered whether the scale for the Cauchy prior is affected by centering the design matrix (I should say that I don't fully understand the process of projecting the matrix into the lower dimensional space and its implications). As discussed in the paper (p. 363), in case of two groups in which groups are coded as 0.5 and -0.5, the projected matrix yields a coding of sqrt(2)/2 and -sqrt(2)/2. As a result, the effect is not coded as a distance of 1 but rather as a distance of sqrt(2). Unless I misunderstand, this implies that the Cauchy prior for fixed effects in the ANOVA needs to be scaled accordingly; otherwise r = 1 would imply a different prior in the case of the t test (where no design matrix is centered) and the ANOVA.

I tried this in the BayesFactor package and it appears that this is indeed the case:

```
library("BayesFactor")
library("dplyr")
medium_scale <- sqrt(2) / 2
wide_scale <- 1
ultrawide_scale <- sqrt(2)
ttest_medium <- ttestBF(formula = extra ~ group, data = sleep, rscale = medium_scale)
ttest_wide <- ttestBF(formula = extra ~ group, data = sleep, rscale = wide_scale)
ttest_ultrawide <- ttestBF(formula = extra ~ group, data = sleep, rscale = ultrawide_scale)
# Different results despite same r
ttest_medium %>%
as.vector ==
anovaBF(formula = extra ~ group, data = sleep, rscaleFixed = medium_scale) %>%
as.vector
ttest_wide %>%
as.vector ==
anovaBF(formula = extra ~ group, data = sleep, rscaleFixed = wide_scale) %>%
as.vector
ttest_ultrawide %>%
as.vector ==
anovaBF(formula = extra ~ group, data = sleep, rscaleFixed = ultrawide_scale) %>%
as.vector
# Same results when r is scaled to fit projected design matrix
distance <- BayesFactor:::fixedFromRandomProjection(nlevRandom = 2) %>%
as.vector %>%
diff
ttest_medium %>%
as.vector ==
anovaBF(formula = extra ~ group, data = sleep, rscaleFixed = medium_scale / distance) %>%
as.vector
ttest_wide %>%
as.vector ==
anovaBF(formula = extra ~ group, data = sleep, rscaleFixed = wide_scale / distance) %>%
as.vector
ttest_ultrawide %>%
as.vector ==
anovaBF(formula = extra ~ group, data = sleep, rscaleFixed = ultrawide_scale / distance) %>%
as.vector
```

I find this quite confusing, as I was under the impression that the scale of the (multivariate) Cauchy in the ANOVA case was similarly placed on the scale of standardized mean differences (d) for each effect. However, from toying around with this example I get the impression that the prior is placed on scaled d units. Is this correct? Does the centering of the matrix cause similar changes to the scale of the prior for each mean difference within a factor? How do I correct for them, if I want to use a consistent prior across analyses?

Thanks in advance,

Frederik

So I have been wondering something lately; is it possible to have JASP give you some sort of indicator of the effect for an AN(C)OVA, or t-test? I'm looking for some sort of Bayesian equivalent of partial eta squared or Cohen's d. Would be amazing if there was a paper somewhere with guidelines on how to qualitatively interpret that Bayesian effect size as small, medium or large.

]]>Therefore, the available models of interest are:

group

IQ

group +IQ

How do you interpret the relationship between the different BFs (or BF inclusion, analysis of effects) for each model? If the BF for group + IQ is more than 3x the group BF (or the BFinclusion > 3) what does this mean for how the group BF should be interpreted?

Many thanks in advance for any help!

]]>Can I add random slopes to the lmBF function?

Currently I'm using the following to test a model with 2 main effects.

lmBF(RT ~ IV1 + IV2 + Subject, whichRandom="Subject",data = RTData)

I want to add the slopes of one or both IVs as a random factor. Is there such a possibility?

Thanks,

Alon

The more I read in the forum, the more confused i get. With a simple design, results are still manageable. For t tests, one can report BF10 or BF01, done. For a simple 2x2, one can describe the model that provides the best evidence (maybe its for a main effect, maybe its for an interaction, maybe both are good), one can report the best or various. Alternatively, if the BF10 is arbitrary, one can look at BF01. Report BF01 instead, done. (Maybe I remember wrong, but in the workshop I believe it was said that the model comparison is whats better than just looking at effects and reporting BFincl?? though for simple designs it wouldnt make much difference, right?).

For a more complicated design, which results in e.g. 166 models, it becomes more tricky.

From the workshop, I remembered to look at the models first, compare to null. That might very straightforward identify that the model including a main effect is best, the others arent even comparable, wonderful, I'll report that BF10 for that model, maybe it even is consistent with my p value, winning.

But with 166 models, I might have a lot of them providing extreme evidence. So I can then compare to the best model, and see whether they really are as good as the best model against the null. But maybe they are, so then I need to understand what drives these models (many containing interactions).

So from the forum, it appears I should go into the output for effects. So here, I can look at effects across all models, hopefully that identifies some main effects. If so, wonderful, I'll report the BFincl for whatever effects win. I might or might not have a strong BFincl for any many effect, but importantly I have evidence for an interaction. Though, this is not yet real evidence for the interaction as in this analyses the models that contain an interaction also contain the main effects. So to evaluate the interaction, I need to look at the output of effects across matched models (or Bawes) - so this strips away the effect from the models.

Thus, this analysis wont ever provide strong BFincl for main effects (since they were stripped, correct), but instead allows me to understand whether the interaction exists and wasnt just driven by a main effect, correct?

So having identified maybe some main effects, some interactions, do I then stick to the BFincl, or should I return to the model comparison?

I guess my confusion comes from my various data sets:

1) While for one main effect I get a BF10=80, when I look at BF01 instead, various models (incl interactions) provide extreme BF01. When I look at analysis of effects across all models, that main effect has a BFincl=13. When I look at analysis of effects across matched models, that main effect has a BFincl=90, while the interactions have weak BFincl. So the interactions in BF01 were driven be the main effect. But how is my BFincl for the main effect now 90 if in this Bawes analyis the models are stripped of the effect? So I'd say the strongest evidence is for that main effect, reporting that model BF10, but do I also report the BFincl, and which one of the two?

2) On a different data set, again 166 models, my BF10 are weak, but the BF01 for main effects =10 and various other models are extreme. Here analysis of effects across all models show only effect of the fourway interaction with BFincl = 743. So again, to evaluate the interaction I look at effects across matched models, and here beautifully, that interaction turns into a BFincl = 1.743E+8. So though I actually have a strong BF01 (though only =10) for the null regarding main effects, there is extreme BFincl for that interaction. So I should just report the evidence for the interaction, I feel like I'm then not telling the whole story here?

3) In another data set, again 166, I have a wonderful BF10 for the model incl factor1 + factor2 + factor1*factor2. In effects across all models, each of the BFincl tell the same story, both factors = 2.8E+13 and their interaction = 2E+14. When I strip away the effect across matched models, the interaction comes out nicely with BFincl = 9.3E+15. So this all makes sense to me, each factor has strong evidence, and their interaction is there too.

But again, do I report the model, the BFincl across models or across matched models?

Sorry about the essay, I just really want to get this right.

MANY MANY THANKS!

Best,

Clarisse

I have conducted several bayesian linear regressions and was looking for an example of how to report in manuscript. Are there any clear examples or guidelines available?

Thanks

Andrew ]]>

I am currently getting into using BF on my data. I would like to ask you to share some experimental papers that implement BF in their analyses. I'd like to get more acquiented with 'how it's done' in practice - how people use and report BF in recent publications.

Best,

Wojciech

I was wondering what options there may be to create priors in an RM-ANOVA, which have been informed from existing data. I had collected data in an initial experiment, but then replicated the study on a much larger scale. I am hoping to use the data from the first study to informed the priors to be be used in the analyse in the follow-up study.

Kind regards,

Michael

]]>I analyzed my data with JASP, submitted it, and now the reviewers require of me to do the analyses again but with different priors to check the robustness for my Bayesian ANOVAs. I know that this is easily done with Bayesian t-tests, but could not find how I can do this for the ANOVAs. I think it would be sufficient if I could run the ANOVAs again, but with different priors (medium, wide, and ultrawide). I see that in the advanced options under 'prior' I can adjust 'r-scale fixed effects' and 'r-scale random effects'. I assume these have to do something with what I am trying to achieve, but am not sure which numbers I have fill out to achieve medium/wide/ultrawide priors. Some help would be much appreciated!

Kevin

]]>Calculating the Bayes Factors for all possible models compared to the null model, it shows me bayes factors of 50 million big? Is that possible? ]]>

Thank you for the great work on the program and adding the BFinclusion factor for RM ANOVAs. Because I was used to calculating a Bayes Factor for an interaction effect by hand (according to the "Bayes like a Baws" post), I did this first and then realized that the new BFinclusion features exists. Interestingly, I get different values for the same interaction effect with both methods. Therefore, I wanted to ask what the exact differences between both calculations are and which one might be more suitable or easier to interpret?

Thanks for the help!

]]>Best regards

Ester

One-sided (>0) Bayesian One Sample T-Test: d BF₊₀ = 2485, error % NaN

Two-sided Bayesian One Sample T-Test: d BF₁₀ = 1242, error % 5.005e -6

]]>Thank you!

Can anyone ove this JASP forum please

I run into a problem with the repeated measures anova. I have done a pilot, collecting data 5 days in a row. They had to solve the same problems each day in order to train them on the problems. Of course, I am interested in seeing if their reaction time for solving the problems lowered during their training. My data is organized in the following way: I have a column with subject number, a column with RT_1, for the reaction time for each problem individually for session one, RT_2, for session 2 and so on. The thing is, RT is only interesting when problems are solved correctly. So, when a problem was solved incorrect, the value is empty.

Now, Jasp only included the problems that are solved correctly over all 5 sessions, because when for a problem (one for each row), one value is missing, it also excludes the RT in the other sessions when the problem was solved correctly. Can I force Jasp to include all of the values, independent of how they were solved in another session?

I cannot find this exact question in other topics considering missing values, but sorry if this question already has been answered elsewhere. I just could not find it.

Kind regards

]]>JASP is great, thanks for developing it! An issue that has persistent over the past and current versions occurs when the number of cells in a repeated-measures ANOVA is relatively high.

When running a 3x9 rm-ANOVA, I get the following in JASP 0.8.2:

Error message, including stack trace:

This analysis terminated unexpectedly. Error in complete.cases(x, y): not all arguments have the same length Stack trace analysis(dataset = NULL, options = options, perform = perform, callback = the.callback, state = state) .resultsPostHoc(referenceGrid, options, dataset, fullModel) t.test(unlist(postHocData[listVarNamesToLevel[[k]]]), unlist(postHocData[listVarNamesToLevel[[i]]]), paired = T, var.equal = F) t.test.default(unlist(postHocData[listVarNamesToLevel[[k]]]), unlist(postHocData[listVarNamesToLevel[[i]]]), paired = T, var.equal = F) complete.cases(x, y) To receive assistance with this problem, please report the message above at: https://jasp-stats.org/bug-reports

The message in 0.7.1.12 used to be `Invalid comparison with complex values`

, and has been reported on GitHub before (raised in September 2015; closed in March 2017 in anticipation of whether it was still a bug in the latest version).

Please note that the issue does not seem to be with the data: Running 2x9 rm-ANOVAs on the same data does work, and I've had the same issue with a completely different dataset. The only common factor seems to be the number of cells.

**UPDATE (2017-09-14, 12:34 UTC+0)**: The same data does work with 2x9 rm-ANOVAs on JASP 0.7.1.12, but not on 0.8.2. On 0.8.2 it does work in a 3x6 design.

Any idea what might be going on?

Cheers,

Edwin

]]>I just want to make sure I'm reporting it correctly in my write up.

Cheers,

]]>I want to find the BF for the main effect of A, but the best model [18] contains also two interactions (A:C and A:B) and the best model without A (4) has no interactions. If I comparte 18 against 4 (sort(Anova)[18]/sort(Anova)[4]) I get a BF of 1.587706e+14, but it includes the interactions.

The other option would be 12 (with only main effects) against 4, which gives 1.606811e+12.

```
> sort(Anova)
Bayes factor analysis
[1] B + S : 35319663 ±0.41%
[2] C + S : 35932312 ±0.2%
[3] C + B + C:B + S : 1.107802e+13 ±0.81%
[4] C + B + S : 9.975825e+13 ±0.58%
[5] A + S : 1.510931e+15 ±0.31%
[6] A + C + S : 2.025998e+18 ±1.27%
[7] A + C + A:C + S : 1.408867e+19 ±0.76%
[8] A + C + B + A:B + C:B + S : 1.098129e+25 ±4.21%
[9] A + C + B + C:B + S : 1.935634e+25 ±0.93%
[10] A + C + B + A:B + S : 7.503825e+25 ±3.19%
[11] A + B + A:B + S : 9.598767e+25 ±1.01%
[12] A + C + B + S : 1.602926e+26 ±1.86%
[13] A + C + A:C + B + C:B + S : 2.149663e+26 ±1.14%
[14] A + B + S : 2.969097e+26 ±1.38%
[15] A + C + A:C + B + A:B + C:B + A:C:B + S : 6.400354e+26 ±1.47%
[16] A + C + A:C + B + S : 1.63816e+27 ±1.27%
[17] A + C + A:C + B + A:B + C:B + S : 2.496953e+27 ±1.14%
[18] A + C + A:C + B + A:B + S : 1.583868e+28 ±0.84%
```

Then to find the BF of A:B I would compare 18/16, and for A:C it would be 18/10?

Which option is better and why? Do I need to use my best model to find the BF of each main effect and interaction, or can I compare two models that allow me to make a fair comparison even if I don´t use the best model?

Thanks.

]]>Thanks ]]>

Here is an example of three different methods (fixed effects, random effects) for analyzing Models in R Commander ]]>

I recently conducted an experiment and followed it with a replication. Both experiments include 2 within-subject conditions. I conducted a Bayesian dependent sample t-test using JASP on each one of them separately and found the for the first experiment BF10 = 0.505 and for the second experiment BF10 = 0.603. I saw in a recent paper that in order to update the BF after the replication I should just multiply the two BF (which would give me BF10 = 0.505*0.642 = 0.304). At the same article it was mentioned that the same result should be observed if I combine the two data sets to a single data set. When I tried that, I found a BF10 of 1.131. Perhaps I'm missing something very trivial?

Thanks a lot for all your help!

Alon

As a new learner of Jasp and Bayes analyses, I re-analyzed some of my old data where the overall performance of my participants was slightly but significantly above chance (>50%; unilateral one sample t-test; t(23)=2.14,p=0.02).

Regarding unilateral Bayesian T-test, I have a Bayes Factor of 2.8 when using the default value of Cauchy prior width (and my Bayes Factor increases, reaching more than the value 3, when I decrease the prior).

I am a bit concerned regarding the conclusions I should draw on my data:

1/ is the Bayes Factor low enough so that I should re-interpret my data and do not conclude in favor of H1 ?

2/ About the manipulation of the prior, I wonder what are the consequences of increasing/decreasing this factor?

3/ I am not sure about the information in the Sequential Analysis plot, could someone give me hints about how to read it ?

Thanks a lot,

JulianeH

]]>