Determining the Best Model in JASP

Comments

• uhandoko

  March 2019

  Output.docx

  This is the output
• EJ

  March 2019

  Hi Uhandoko,

  
  

  We are finishing up a tutorial paper on this. I have asked the first author to send you a draft. Quickly though:

  1. If you tick "compare to best model", the models will be arrange in decreasing order of predictive performance.
  2. If you want to predict anything, the Bayesian course of action is to average over the parameters (so use the full posterior distribution, not just the point estimate) *and* average over the models (so do not use a single model).

  Cheers,

  E.J.
• uhandoko

  March 2019

  Output.docx

  Thank you very much EJ for your quick response and your kindness will send me a draft about this tutorial.

  Sorry, I still don't understand the second answer. As I understand from your answer to predict something with Bayesian regression equation,  the coefficients regression for each variable  using  "mean" column in the posterior summary table. For example, from the output I get, the first ranking model consisted of 12 climate models. So to predict rain, I will use the equation as follows:

  Rainfall prediction = 8.687 + (- 0.450 * NorESM) + (- 0.299 * ACCESS) + (- 0.082 * bcc) + (- 0.085 * BNU) + (0.905 * CanESm2) + ... + (0.02 * MRI).

  Is it like that?

  
  

  Best regard..

  Uhandoko
• EJ

  March 2019

  Dear Uhandoko,

  There is a whole literature on how to predict exactly. In general, I would say you want the uncertainty surrounding your point prediction, so you'd want to take the distributions for the beta's into account (rather than just focusing on the means). Basically, I would do the following:

  1. For each of the models k=1...K, sample from the joint posterior distribution and make a prediction.
  2. The number of samples from each model should be proportional to their posterior probability.

  This way you take into account uncertainty in the beta's and in the models. But whatever you do, both need to be taken into account or else the prediction will be overconfident.

  Cheers,

  E.J.
• uhandoko

  March 2019

  Thank you for the advice for me, EJ

  
  

  Best regard...

  Uhandoko