Predictions with regression tools
Hello to everybody,
I have tried to predict quantitative variables with JASP but I run into two problems.
The first is that the "Regression" tool gives you only regression coefficients, their confidence limits and significance information, but does not produce either predicted values or prediction statistics (RMSEP, Determination coefficent and so on). At least, I was not able to obtain them.
For getting predictions I had to use the "Machine Learning" tool, but again I run into a problem. I tried two different tools (decision trees and SVM), but I got predicted valueas only in centered ( or, maybe, auto-scaled form) and not rescaled for matching the calibration data. Moreover, it seems that centered values are also used for calculating evaluation statistics (RMSE, MAE, MAPE).
This bring abnormally high values of MAPE (Mean Absolute Percent Error), because when the centered Y variable crosses 0 the denominator of MAPE formula diverge.
MAPE = 100 x mean( abs ( (Y - Yh) / Y ) ), where Yh is the prediction and Y the reference value.
Is this a bug of JASP or did I forget to set some option in the Machine Learning dialogue box?
Best regards.
Leonardo_C
Comments
This might not be ideal but as a temporary fix in the machine learning module you can uncheck "Scale variables" under "Training Parameters" (both for decision tree and SVM)
For the regression module you get the coefficient of determination (i.e. R²) and RMSE by default (first table in output). You can't split your training set in this module.
Yes the best way to go is to uncheck the ‘scale predictors’ box. This way the raw data is used for everything.
If you are missing a feature in the evaluation metrics table, could you please suggest it via out github page: https://github.com/jasp-stats/jasp-issues/issues? Thanks!
OK. i will try this way. Thanks.
Leonardo_C