Correction for Dependency - Meta-Analysis

Max123

Hey Everyone,

I couldn´t find it, and was thus wondering whether JASP allows me to correct for dependency when conducting a meta-analysis.

Best,

Max

EJ

Hi Max,

I'll direct your question to our meta-analysis expert. Just for my own education: what exactly do you mean with "dependency"?

Cheers, E.J.

Max123

Hey E.J.,

Thank you for your help!

I think I need to give a bit more concrete background:

A recent paper found that there are many different techniques for data pre-processing (e.g. trimming/log-transformation) used in my subfield. The authors showed that those different pre-processing techniques had profound influence on whether one sample dataset showed "significant" effects, even when using more complex analyses such as Bayesian analyses or LLMs.

I planned on re-analysing existing studies using each of those preprocessing techniques and then compare the mean effect sizes of each preprocessing pathway. That means, I have one effect size per pre-processing pathway, each of which is build on the same data (e.g. 4 effect sizes per study). So if I were to run a sub-group meta-analysis with the preprocessing pathway as group-factor, then the effect sizes in the study would not be independent of each other. Simply put, dependency here means that (some of) the effect sizes are based on the same studies.

Thank you for your time!

Best,

Max

Raoul

Hi Max,

The meta-analysis module in the current version of Jasp does not yet allow for this. The next release probably will allow for what is sometimes called 'multivariate meta-analysis'. If you need to get to it right away, I recommend to have a look at the rma.mv() examples in the metafor package for R.

Hope that helps!

Best,

Raoul

Max123

Dear Raoul,

thank you for your response! Unfortunately, I did not work with R yet, but I will see if I have time to look into it. When is the next release scheduled?

Thank you for your time!

Best,

Max

EJ

The next version is only weeks away

Cheers,

E.J.

Nancysara_123

There are several software packages used for clinical meta-analysis, including MetaFOR library for R, JASP, comprehensive meta-analysis, Revman, etc. But, these types of software packages are only adapted to basic science projects. For the analysis of large and complex datasets, these software packages may not be good. Recently, MATLAB R2016b software, introduced by MetaLab, is useful for various statistical analyses in meta-analysis.

EJ

Hi Nancysara_123,

Please let us know on our GitHub page what meta-analysis features you'd like JASP to have! We aim to make students and researchers less dependent on commercial software. (for details see https://jasp-stats.org/2018/03/29/request-feature-report-bug-jasp/).

Cheers,

E.J.

Nancysara_123

Meta-analysis refers to the statistical synthesis of a variety of studies quantitative findings.  Dependence also occurs commonly when two treatment groups are included in the research design of a study compared with the same control group. Because each treatment / control comparison involves the same control group participants, the resulting effect sizes are statistically dependent. Failure to overcome or model dependency leads to artificially reduced variance estimates which in turn inflates Type I error. Treating dependent effect sizes as if they were independent often gives more weight to studies that have several measures or more than two classes in the meta-analysis. Statistical dependency must be overcome in such a way that modeling computational techniques to control dependency in order to avoid these challenges to the integrity of the meta-analytical findings or each study will contribute to the meta-analysis by a single independent effect size.

EJ

That's a good suggestion, I'll pass this along.

Cheers,

E.J.

fvasilopoulos

Hi E.J. & Raoul,

Can I double check that the current version of JASP includes multivariate meta-analysis rma.mv() from the metafor package?

Thanks,

Fotini

Raoul

Hi Fotini,

The current version does not yet include rma.mv() unfortunately. Hopefully it will in a near future release when time permits.

Yet, I highly recommend to simply use R and the metafor package to run this more sophisticated analysis (and potentially discover the all the problems that can result fitting a multi-level model that make it harder to handle in a GUI).

Best,

Raoul