Contradicting results in JASP Mediation

Michael_Jasper

For a mediation analysis in JASP (using bias-corrected percentile bootstrapping). 

For the Indirect effect, I have a situation where: 

the p-value is significant (< 0.05), 

BUT where 0 is found within the range of the 95% Confidence Interval. 

So, these contradict. And so, I am unsure what to take from this. 

In this scenario, is there statistical evidence for an indirect (mediation) effect or not? 

EJ

This is strange. In fact, it should not happen, because a 95% CI contains all the values that would not be rejected using an alpha=5% test. The conflict could be due to the noise in the bootstrap...can you provide a concrete example so we can reproduce this?

Cheers,

E.J.

JMBKoch

Hi Michael_Jasper,

This is a good question, and I understand your confusion here. The reason why the results can contradict in this case, is because the p-value comes from a different test, and not form the (bias-corrected percentile) bootstrap. The p-value is based on the standard errors that are computed using the so-called Delta method (this is listed in the footnote under the table). Hence, the s.e. and p-value are the ones from the 'Standard' method. If you want to test your indirect effect using a bias-corrected bootstrap (which I would advise you to do as well, since it is known to outperform the delta method) you should just report the point estimate and the 95% C.I. Since in your case the interval contains zero, you cannot reject your Null-Hypothesis that there is no indirect effect.

Best,

Michael

Michael_Jasper

RE: you said:

"you should just report the point estimate and the 95% C.I.".

Ok, understood. Although if I wanted a p-value from these values, is this possible? If so, how can I get it?

Michael_Jasper

So, I use Percentile bootstap. And report the Estimate and the Confidence Interval (CI). But NOT the given p-value because, as said above, it comes from using a different method.

But if I wanted a p-value for the actual Estimate and CI produced by the Percentile bootstrap method. Would the method described in this paper be appropriate/valid?

https://www.bmj.com/content/343/bmj.d2304