Sharply Value

Faming_Wang

Hi there,

I wonder if there is a way to compute Sharply Value in JASP when I use the machine learning module, such as random forest.

Best

Faming

EJ

Hello Faming,

Do you mean Shapley values? Can you include a link or citation to clarify the concept you are referring to?

Cheers,

E.J.

koenderks

There is a way to see the relative impact of each feature in a machine learning model by clicking “explain predictions” in the interface. Under the hood, this uses the breakdown algorithm (https://ema.drwhy.ai/breakDown.html) instead of the shap algorithm (https://ema.drwhy.ai/shapley.html) but the goal is the same.

TarandeepKang

Hi Koen,

Regrettably, I think I found a bug with the implementation of explain predictions for random forest regression.

[Bug]: explain predictions not working for random forest regression · Issue #2731 · jasp-stats/jasp-issues (github.com)

I was experimenting with this feature before using it. I couldn't any documentation in the help file. Please could you consider expanding this aspect (the explain prediction section) of it?

Best,

Tarandeep

koenderks

It is briefly mentioned under the "Input --> Tables" section as "Explain predictions: Shows the decomposition of the model’s prediction into contributions that can be attributed to different explanatory variables". When clicking it, you will see the following table.

This table basically dissects the prediction of a test set case into a 'function' of its features. For instance, using the album sales dataset and a random forest regression model, the predicted sales for the first test set case are 137.977. This is explained by a "base prediction" of 190.289. The value for adverts of this first case results in lowering the base prediction by around 11 sales, the value of airplay lowers it by an additional 31 sales and its value for attract lowers it by another 9.9 sales. Hence, 190.289 - 11.316 - 31.017 - 9.989 = 137.967.

TarandeepKang

Hi Koen,

My apologies, it seems I might have been unclear. I rather meant, perhaps it might be useful if you might add some more references to the details of this particular algorithm in the help file.

Of particular use would be a clarification of whether JASP uses explainpredidiction or ibreakdown for this calculation together with the relevant citation the method.

Do you want me to open an issue?

Best,

Tarandeep

koenderks

No need, thanks! I've added some lines to the help files in an existing pull request.

Faming_Wang

Dear E.J.,

Sorry for the typo. Yes, it should be Shapley values. Thanks for Koen's response.

Best

Faming