The metafor Package

A Meta-Analysis Package for R

Sidebar

analyses:lipsey2001

Lipsey & Wilson (2001)

The Methods and Data

The book Practical meta-analysis by Lipsey and Wilson (2001) is an excellent introduction to meta-analysis and covers in less than 250 pages the most important aspects of conducting a meta-analysis. Here, I will reproduce the analyses conducted in chapter 7, which is focused on the computational aspects of a meta-analysis, including the fixed-, random-, and the mixed-effects model. The data are provided in the book on page 130 in Table 7.1. We can create the same dataset with:

dat <- data.frame(
id = c(100, 308, 1596, 2479, 9021, 9028, 161, 172, 537, 7049),
yi = c(-0.33, 0.32, 0.39, 0.31, 0.17, 0.64, -0.33, 0.15, -0.02, 0.00),
vi = c(0.084, 0.035, 0.017, 0.034, 0.072, 0.117, 0.102, 0.093, 0.012, 0.067),
random = c(0, 0, 0, 0, 0, 0, 1, 1, 1, 1),
intensity = c(7, 3, 7, 5, 7, 7, 4, 4, 5, 6))

The contents of dat are then:

     id    yi    vi random intensity
1   100 -0.33 0.084      0         7
2   308  0.32 0.035      0         3
3  1596  0.39 0.017      0         7
4  2479  0.31 0.034      0         5
5  9021  0.17 0.072      0         7
6  9028  0.64 0.117      0         7
7   161 -0.33 0.102      1         4
8   172  0.15 0.093      1         4
9   537 -0.02 0.012      1         5
10 7049  0.00 0.067      1         6

Fixed-Effects Model

After loading the metafor package with library(metafor), we can fit a fixed-effects model to these data with:

res <- rma(yi, vi, data=dat, method="FE")
res
Fixed-Effects Model (k = 10)

Test for Heterogeneity:
Q(df = 9) = 14.7640, p-val = 0.0976

Model Results:

estimate       se     zval     pval    ci.lb    ci.ub
0.1549   0.0609   2.5450   0.0109   0.0356   0.2742        *

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

These results match what is reported on page 132-133 (see Exhibit 7.3).

Random-Effects Model

A random-effects model (using the DerSimonian-Laird estimator) can be fitted with:

res <- rma(yi, vi, data=dat, method="DL")
res
Random-Effects Model (k = 10; tau^2 estimator: DL)

tau^2 (estimated amount of total heterogeneity): 0.0260 (SE = 0.0328)
tau (square root of estimated tau^2 value):      0.1611
I^2 (total heterogeneity / total variability):   39.04%
H^2 (total variability / sampling variability):  1.64

Test for Heterogeneity:
Q(df = 9) = 14.7640, p-val = 0.0976

Model Results:

estimate       se     zval     pval    ci.lb    ci.ub
0.1534   0.0858   1.7893   0.0736  -0.0146   0.3215        .

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Again, these results match what is shown in Exhibit 7.3 and on pages 134-135.

Analog to the ANOVA

Next, we can carry out the ANOVA-type analysis described on pages 135-138. In particular, a fixed-effects model with the random variable as moderator can be fitted with:

res <- rma(yi, vi, mods = ~ random, data=dat, method="FE")
res
Fixed-Effects with Moderators Model (k = 10)

Test for Residual Heterogeneity:
QE(df = 8) = 7.6901, p-val = 0.4643

Test of Moderators (coefficient(s) 2):
QM(df = 1) = 7.0739, p-val = 0.0078

Model Results:

estimate      se     zval    pval    ci.lb    ci.ub
intrcpt    0.2984  0.0813   3.6687  0.0002   0.1390   0.4578  ***
random    -0.3261  0.1226  -2.6597  0.0078  -0.5664  -0.0858   **

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Note that the $Q_E$ and $Q_M$ statistics add up to the $Q$ statistic given earlier for the fixed/random-effects models (i.e., $7.6901 + 7.0739 = 14.764$).

The tests for heterogeneity within the two subgroups formed by the moderator can be obtained with:

rma(yi, vi, data=dat, method="FE", subset=random==0)
Fixed-Effects Model (k = 6)

Test for Heterogeneity:
Q(df = 5) = 6.4382, p-val = 0.2659

Model Results:

estimate       se     zval     pval    ci.lb    ci.ub
0.2984   0.0813   3.6687   0.0002   0.1390   0.4578      ***

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

and

rma(yi, vi, data=dat, method="FE", subset=random==1)
Fixed-Effects Model (k = 4)

Test for Heterogeneity:
Q(df = 3) = 1.2519, p-val = 0.7406

Model Results:

estimate       se     zval     pval    ci.lb    ci.ub
-0.0277   0.0917  -0.3017   0.7628  -0.2075   0.1521

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Here, we note that the two separate $Q$ statistics add up to the $Q_E$ statistic given earlier (i.e., $6.4382 + 1.2519 = 7.6901$).

Finally, the predicted/estimated effect for random = 0 and random = 1 can be obtained with:

predict(res, newmods=c(0,1))
     pred     se   ci.lb  ci.ub
1  0.2984 0.0813  0.1390 0.4578
2 -0.0277 0.0917 -0.2075 0.1521

All of the values shown in Exhibit 7.4 have now been computed.

Weighted Regression Analysis

The "weighted regression analysis" described on pages 138-140 is again nothing else than a fixed-effects model with moderators. Here, the authors are including both the random and the intensity moderators in the model. This can be done with:

res <- rma(yi, vi, mods = ~ random + intensity, data=dat, method="FE")
res
Fixed-Effects with Moderators Model (k = 10)

Test for Residual Heterogeneity:
QE(df = 7) = 7.6832, p-val = 0.3614

Test of Moderators (coefficient(s) 2,3):
QM(df = 2) = 7.0807, p-val = 0.0290

Model Results:

estimate      se     zval    pval    ci.lb    ci.ub
intrcpt      0.3223  0.2998   1.0752  0.2823  -0.2652   0.9099
random      -0.3298  0.1304  -2.5286  0.0115  -0.5854  -0.0742  *
intensity   -0.0041  0.0493  -0.0829  0.9339  -0.1007   0.0925

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Exhibit 7.6 (page 141) shows the same results.

Mixed-Effects Model

Finally, the mixed-effects model is described on pages 140-142. These results can be obtained with:

res <- rma(yi, vi, mods = ~ random + intensity, data=dat, method="DL")
res
Mixed-Effects Model (k = 10; tau^2 estimator: DL)

tau^2 (estimated amount of residual heterogeneity):     0.0049 (SE = 0.0294)
tau (square root of estimated tau^2 value):             0.0698
I^2 (residual heterogeneity / unaccounted variability): 8.89%
H^2 (unaccounted variability / sampling variability):   1.10
R^2 (amount of heterogeneity accounted for):            81.20%

Test for Residual Heterogeneity:
QE(df = 7) = 7.6832, p-val = 0.3614

Test of Moderators (coefficient(s) 2,3):
QM(df = 2) = 5.5711, p-val = 0.0617

Model Results:

estimate      se     zval    pval    ci.lb    ci.ub
intrcpt      0.3311  0.3198   1.0351  0.3006  -0.2958   0.9579
random      -0.3269  0.1439  -2.2712  0.0231  -0.6090  -0.0448  *
intensity   -0.0068  0.0528  -0.1292  0.8972  -0.1103   0.0967

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

These are the same results as shown in Exhibit 7.7 (page 141).

Heterogeneity Accounted For

The amount of heterogeneity accounted for by the moderator(s) included in the model is typically computed by comparing the amount of heterogeneity from the random-effects model with the amount of residual heterogeneity from the mixed-effects model (for more details, see the FAQs). The value is given in the output above. We can also carry out this computation explicitly with:

res.ME <- rma(yi, vi, mods = ~ random + intensity, data=dat, method="DL")
res.RE <- rma(yi, vi, data=dat, method="DL")
anova(res.RE, res.ME)
        df    AIC    BIC  logLik    LRT   pval      QE  tau^2   R^2
Full     4 4.9040 6.1143  1.5480                7.6832 0.0049
Reduced  2 6.1347 6.7399 -1.0674 5.2308 0.0731 14.7640 0.0260 81.2%

The value under R^2 is the amount of variance (heterogeneity) accounted for. In this particular example, we estimate that 81.2% of the total amount of heterogeneity is accounted for by the two moderators. Note that Lipsey and Wilson calculate $R^2$ in a different way than is usually done in meta-analyses, so the value for $R^2$ shown in Exhibit 7.7 (page 141) is different from the value computed above.

References

Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.