Prior distribution regression analyses // different results using frequentist and Bayesian inference

alexa

Hello everybody,

I got the chance to revise the first paper in which I used Bayesian inference. It’s quite a while that I have been working with Bayesian inference. Now, having a little distance to my analyses, I am a little bit confused regarding the prior distribution and I am wondering if I have done a mistake in my analyses.

First, I only computed simple correlation analyses to present the correlations between the constructs (let me say construct A and construct B). I used the default prior distribution. Is that correct? 

Second, I computed a regression analysis. Based on theory, I hypothesized that construct A will predict construct B. Again, as there were no previous data on the relationship between A and B, I used the default prior model probabilities (1/2 = 0.5). Now I am a little bit confused if I should have changed the prior width before conducting the analysis?

And one last question: I used both frequentist and Bayesian methods of inference. In a further exploratory regression analysis, I tested if construct A with four predictors (A1/A2/A3/A4) predicted construct B. Bayesian regression results showed that the model with the predictors A2 and A3 outperformed all other models. However, classical regression results did not reveal any significant predictors. One reviewer asked me to explain this result and I am wondering how to do that.

I would be very happy if someone could help me out!

Thank you very much!

Alexa

EJ

Hi Alexa,

1. The correlation with the default (uniform) parameter prior under H1: I have no problems with it, and it is what Jeffreys proposed. But you might try to make H1 directional (you probably have a direction in mind? either a positive or a negative correlation?)
2. The regression analysis: the relevant prior for the Bayes factor analysis is the prior on the parameter values under H1; the prior on the models themselves is not part of the Bayes factor. The default setting will produce slightly different BFs than the uniform from the correlation analysis.
3. Your last question is difficult to assess without more information. I would have to see an output table of some sort, ideally for both the classical and the Bayesian approach.

Cheers, E.J.

alexa

E.J., thank you very much for this helpful and fast answer.

1. / 2. Ah, I understand. Actually, I wanted to present the correlation table just as “an overview” (as I assessed several constructs), but I didn’t want to test any hypothesis. Although, of cause, I assumed e.g., a positive relationship (directional H1) between A and B and a negative relationship between A and C (directional H1). The testing of the association between other constructs, e.g., between A and D was exploratory (uniform). Afterwards, I conducted the regression analysis (linear and multiple regression analyses) with the default prior (directional H1 as you said). (I hope that my description is understandable). Do you think in my case it is “OK” to conduct both correlation and regression analyses with the default prior or should I adjust the correlation analysis? I just quickly re-ran the correlation analysis and you are right, there are slightly different BF when testing a directional H1.

3. I will send you a part of the Tables via PM (because of unpublished data), I hope that this is OK.

Thank you!!