tips:i2_multilevel_multivariate

This shows you the differences between two versions of the page.

Both sides previous revision Previous revision Next revision | Previous revision | ||

tips:i2_multilevel_multivariate [2018/12/08 13:25] Wolfgang Viechtbauer |
tips:i2_multilevel_multivariate [2019/05/15 19:10] Wolfgang Viechtbauer |
||
---|---|---|---|

Line 3: | Line 3: | ||

The $I^2$ statistic was introduced by Higgins and Thompson in their seminal 2002 paper and has become a rather popular statistic to report in meta-analyses, as it facilitates the interpretation of the amount of heterogeneity present in a given dataset. | The $I^2$ statistic was introduced by Higgins and Thompson in their seminal 2002 paper and has become a rather popular statistic to report in meta-analyses, as it facilitates the interpretation of the amount of heterogeneity present in a given dataset. | ||

- | For a standard random-effects models, the $I^2$ statistic is computed with $$I^2 = 100\% \times \frac{\hat{\tau}^2}{\hat{\tau}^2 + s^2},$$ where $\hat{\tau}^2$ is the estimated value of $\tau^2$ and $$s^2 = \frac{(k-1) \sum w_i}{(\sum w_i)^2 - \sum w_i^2},$$ where $w_i = 1/v_i$ is the inverse of the sampling variance of the $i^{th}$ study. The equation for $s^2$ is equation 9 in Higgins & Thompson (2002) and can be regarded as the 'typical' within-study (or sampling) variance of the observed effect sizes or outcomes.((There are also other suggestions in the literature for defining the 'typical' or 'average' sampling variance of the estimates. For example, Takkouche et al. (1999, 2013) suggest to use the harmonic mean of the sampling variances in an analogous computation of an $I^2$-type statistic, but this never really caught on in practice.)) | + | For a standard random-effects models, the $I^2$ statistic is computed with $$I^2 = 100\% \times \frac{\hat{\tau}^2}{\hat{\tau}^2 + \tilde{v}},$$ where $\hat{\tau}^2$ is the estimated value of $\tau^2$ and $$\tilde{v} = \frac{(k-1) \sum w_i}{(\sum w_i)^2 - \sum w_i^2},$$ where $w_i = 1/v_i$ is the inverse of the sampling variance of the $i^{th}$ study. The equation for $\tilde{v}$ is equation 9 in Higgins & Thompson (2002) and can be regarded as the 'typical' within-study (or sampling) variance of the observed effect sizes or outcomes.((There are also other suggestions in the literature for defining the 'typical' or 'average' sampling variance of the estimates. For example, Takkouche et al. (1999, 2013) suggest to use the harmonic mean of the sampling variances in an analogous computation of an $I^2$-type statistic, but this never really caught on in practice.)) |

- | **Sidenote**: As the equation above shows, $I^2$ estimates the amount of heterogeneity //relative// to the total amount of variance in the observed effects or outcomes (which is composed of the variance in the true effects, that is, $\hat{\tau}^2$, plus sampling variance, that is, $s^2$). Therefore, it is not an //absolute// measure of heterogeneity and should not be interpreted as such. For example, a practically/clinically irrelevant amount of heterogeneity (i.e., variance in the true effects) could lead to a large $I^2$ value if all of the studies are very large (in which case $s^2$ will be small). Conversely, when all of the studies are small (in which case $s^2$ will be large), $I^2$ may still be small, even if there are large differences in the size of the true effects. See also chapter 16 in Borenstein et al. (2009), which discusses this idea very nicely. | + | **Sidenote**: As the equation above shows, $I^2$ estimates the amount of heterogeneity //relative// to the total amount of variance in the observed effects or outcomes (which is composed of the variance in the true effects, that is, $\hat{\tau}^2$, plus sampling variance, that is, $\tilde{v}$). Therefore, it is not an //absolute// measure of heterogeneity and should not be interpreted as such. For example, a practically/clinically irrelevant amount of heterogeneity (i.e., variance in the true effects) could lead to a large $I^2$ value if all of the studies are very large (in which case $\tilde{v}$ will be small). Conversely, when all of the studies are small (in which case $\tilde{v}$ will be large), $I^2$ may still be small, even if there are large differences in the size of the true effects. See also chapter 16 in Borenstein et al. (2009), which discusses this idea very nicely. |

However, this caveat aside, $I^2$ is a very useful measure because it directly indicates to what extent heterogeneity contributes to the total variance. In addition, most people find $I^2$ easier to interpret than estimates of $\tau^2$. | However, this caveat aside, $I^2$ is a very useful measure because it directly indicates to what extent heterogeneity contributes to the total variance. In addition, most people find $I^2$ easier to interpret than estimates of $\tau^2$. | ||

Line 27: | Line 27: | ||

k <- res$k | k <- res$k | ||

wi <- 1/dat$vi | wi <- 1/dat$vi | ||

- | s2 <- (k-1) * sum(wi) / (sum(wi)^2 - sum(wi^2)) | + | vt <- (k-1) * sum(wi) / (sum(wi)^2 - sum(wi^2)) |

- | 100 * res$tau2 / (res$tau2 + s2) | + | 100 * res$tau2 / (res$tau2 + vt) |

</code> | </code> | ||

<code output> | <code output> |

tips/i2_multilevel_multivariate.txt · Last modified: 2019/05/15 19:10 by Wolfgang Viechtbauer

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International