The metafor Package

A Meta-Analysis Package for R

User Tools

Site Tools

faq

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Next revisionBoth sides next revision
faq [2020/06/26 06:56] – Wolfgang Viechtbauerfaq [2021/11/05 09:40] – Wolfgang Viechtbauer
Line 33: Line 33:
 Similar (and much more thorough/extensive) tests have been conducted for the more intricate methods in the package. Similar (and much more thorough/extensive) tests have been conducted for the more intricate methods in the package.
  
-It may also be useful to note that there is now an appreciable user base of the metafor package (the [[https://www.jstatsoft.org/v36/i03/|Viechtbauer (2010)]] article describing the package [[http://scholar.google.nl/scholar?oi=bibs&hl=en&cites=8753688964455559681|has been cited in over 5000 articles]], many of which are applied meta-analyses and/or methodological/statistical papers that have used the metafor package as part of the research). This increases the chances that any bugs would be detected, reported, and corrected.+It may also be useful to note that there is now an appreciable user base of the metafor package. The [[https://www.jstatsoft.org/v36/i03/|Viechtbauer (2010)]] article describing the package [[http://scholar.google.nl/scholar?oi=bibs&hl=en&cites=8753688964455559681|has been cited over 7000 times]]. Many of the citations are from applied meta-analyses and/or methodological/statistical papers that have used the metafor package as part of their research. This increases the chances that any bugs would be detected, reported, and corrected.
  
 Finally, I have become very proficient at hitting the [[https://xkcd.com/323/|Ballmer Peak]]. Finally, I have become very proficient at hitting the [[https://xkcd.com/323/|Ballmer Peak]].
Line 39: Line 39:
 ??? Is the package development funded? ??? Is the package development funded?
  
-!!! For the most part, the development of the package has been funded through my own precious time. Through some collaborative work on the 'Open Meta-Analyst' software from the //Center for Evidence-Based Medicine// at Brown University, I have received some funding as part of a subcontract on a grant. Also, Sandra Wilson and Mark Lipsey of the //Peabody Research Institute// at Vanderbilt University have provided funding that gave me some time and incentive for making the ''rma.mv()'' more efficient and for adding multicore capabilities to the ''profile.rma.mv()'' function. In this context, I also would like to mention Jason Hoeksema from the Department of Biology at the University of Mississippi, who invited me to be part of a working group at the //National Evolutionary Synthesis Center//, which gave me time over the course of several meetings to really develop the capabilities of the ''rma.mv()'' function. This collaboration also helped me better appreciate how important it is to listen to the users' needs when prioritizing additions to the package. However, further developments of the package could proceed much quicker if additional funding was available. If you are aware of any funding possibilities, feel free to let me know!+!!! For the most part, the development of the package has been funded through my own precious time. Through some collaborative work on the 'Open Meta-Analyst' software from the //Center for Evidence-Based Medicine// at Brown University, I have received some funding as part of a subcontract on a grant. Also, Sandra Wilson and Mark Lipsey of the //Peabody Research Institute// at Vanderbilt University have provided funding that gave me some time and incentive for making the ''[[https://wviechtb.github.io/metafor/reference/rma.mv.html|rma.mv()]]'' more efficient and for adding multicore capabilities to the ''[[https://wviechtb.github.io/metafor/reference/profile.rma.html|profile()]]'' functions. In this context, I also would like to thank Jason Hoeksema from the Department of Biology at the University of Mississippi, who invited me to be part of a working group at the //National Evolutionary Synthesis Center// (NESCent), which gave me time over the course of several meetings to really develop the capabilities of the ''[[https://wviechtb.github.io/metafor/reference/rma.mv.html|rma.mv()]]'' function. This collaboration also helped me better appreciate how important it is to listen to the users' needs when prioritizing additions to the package. However, further developments of the package could proceed much quicker if additional funding was available. If you are aware of any funding possibilities, feel free to let me know!
  
 ??? How do I cite the package? ??? How do I cite the package?
Line 45: Line 45:
 !!! First of all, thanks for trying to do so in the first place. The best way of citing the package is to cite the following paper: !!! First of all, thanks for trying to do so in the first place. The best way of citing the package is to cite the following paper:
  
-Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. //Journal of Statistical Software, 36//(3), 1--48.+Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. //Journal of Statistical Software, 36//(3), 1--48. [[https://doi.org/10.18637/jss.v036.i03]]
  
 By the way, try ''citation("metafor")'' in R (this is not a command specific to the metafor package; you can try this with other package names; and ''citation()'' -- without a package name between the parentheses -- will tell you how to cite R itself!). By the way, try ''citation("metafor")'' in R (this is not a command specific to the metafor package; you can try this with other package names; and ''citation()'' -- without a package name between the parentheses -- will tell you how to cite R itself!).
Line 55: Line 55:
 ??? Why can I not just use the lm() and lme(), and lmer() functions to conduct my meta-analysis? ??? Why can I not just use the lm() and lme(), and lmer() functions to conduct my meta-analysis?
  
-!!! First of all, meta-analytic models (as can be fitted with the ''rma()'' and ''rma.mv()'' functions) make different assumptions about the nature of the sampling variances (that indicate the (im)precision of the estimates) compared to models fitted by the ''lm()'', ''lme()'', and ''lmer()'' functions, which assume that the sampling variances are known only up to a proportionality constant. Extra steps must therefore be taken to fix up the output to bring the results in line with standard meta-analytic practices. For more details, I have written up a more comprehensive [[tips:rma_vs_lm_lme_lmer|comparison of the rma() and the lm(), lme(), and lmer() functions]].+!!! First of all, meta-analytic models (as can be fitted with the ''[[https://wviechtb.github.io/metafor/reference/rma.uni.html|rma.uni()]]'' and ''[[https://wviechtb.github.io/metafor/reference/rma.mv.html|rma.mv()]]'' functions) make different assumptions about the nature of the sampling variances (that indicate the (im)precision of the estimates) compared to models fitted by the ''lm()'', ''lme()'', and ''lmer()'' functions, which assume that the sampling variances are known only up to a proportionality constant (when using their ''weights'' arguments). Extra steps must therefore be taken to fix up the output to bring the results in line with standard meta-analytic practices. For more details, I have written up a more comprehensive [[tips:rma_vs_lm_lme_lmer|comparison of the rma() and the lm(), lme(), and lmer() functions]].
  
 Furthermore, there are various additional statistics, analyses, and graphical outputs that typically need to be obtained/reported when conducting a meta-analysis. Therefore, while you could use functions such as ''lm()'', ''lme()'', and ''lmer()'' (possibly with some tweaking) to fit certain types of meta-analytic models, a package such as metafor gives you an integrated toolbox that should cover the entire meta-analytic workflow. Furthermore, there are various additional statistics, analyses, and graphical outputs that typically need to be obtained/reported when conducting a meta-analysis. Therefore, while you could use functions such as ''lm()'', ''lme()'', and ''lmer()'' (possibly with some tweaking) to fit certain types of meta-analytic models, a package such as metafor gives you an integrated toolbox that should cover the entire meta-analytic workflow.
Line 67: Line 67:
 Therefore, depending on the estimator of $\tau^2$ used, the values of $I^2$ and $H^2$ will change. For random-effects models, $I^2$ and $H^2$ are often computed in practice with $I^2 = 100\% \times (Q-(k-1))/Q$ and $H^2 = Q/(k-1)$, where $Q$ denotes the statistic for the test of heterogeneity and $k$ the number of studies (i.e., observed effects or outcomes) included in the meta-analysis. The equations used in the metafor package to compute these statistics are based on more general definitions and have the advantage that the values of $I^2$ and $H^2$ will be consistent with the estimated value of $\tau^2$ (i.e., if $\hat{\tau}^2 = 0$, then $I^2 = 0$ and $H^2 = 1$ and if $\hat{\tau}^2 > 0$, then $I^2 > 0$ and $H^2 > 1$). Therefore, depending on the estimator of $\tau^2$ used, the values of $I^2$ and $H^2$ will change. For random-effects models, $I^2$ and $H^2$ are often computed in practice with $I^2 = 100\% \times (Q-(k-1))/Q$ and $H^2 = Q/(k-1)$, where $Q$ denotes the statistic for the test of heterogeneity and $k$ the number of studies (i.e., observed effects or outcomes) included in the meta-analysis. The equations used in the metafor package to compute these statistics are based on more general definitions and have the advantage that the values of $I^2$ and $H^2$ will be consistent with the estimated value of $\tau^2$ (i.e., if $\hat{\tau}^2 = 0$, then $I^2 = 0$ and $H^2 = 1$ and if $\hat{\tau}^2 > 0$, then $I^2 > 0$ and $H^2 > 1$).
  
-These two sets of equations for $I^2$ and $H^2$ actually coincide when using the DerSimonian-Laird estimator of $\tau^2$ (i.e., the commonly used equations are actually special cases of the more general definitions given above). Therefore, if you prefer the more conventional definitions of these statistics, use ''method="DL"'' when fitting the random/mixed-effects model with the ''rma()'' function.+These two sets of equations for $I^2$ and $H^2$ actually coincide when using the DerSimonian-Laird estimator of $\tau^2$ (i.e., the commonly used equations are actually special cases of the more general definitions given above). Therefore, if you prefer the more conventional definitions of these statistics, use ''method="DL"'' when fitting the random/mixed-effects model with the ''[[https://wviechtb.github.io/metafor/reference/rma.uni.html|rma()]]'' function.
  
 See the analysis example for [[analyses:raudenbush2009|Raudenbush (2009)]] for an example of this. See the analysis example for [[analyses:raudenbush2009|Raudenbush (2009)]] for an example of this.
Line 73: Line 73:
 ??? For mixed-effects models, how is the $R^2$ statistic computed by the rma() function? ??? For mixed-effects models, how is the $R^2$ statistic computed by the rma() function?
  
-!!! The pseudo $R^2$ statistic (Raudenbush, 2009) is computed with $$R^2 = \frac{\hat{\tau}_{RE}^2 - \hat{\tau}_{ME}^2}{\hat{\tau}_{RE}^2},$$ where $\hat{\tau}_{RE}^2$ denotes the estimated value of $\tau^2$ based on the random-effects model (i.e., the total amount of heterogeneity) and $\hat{\tau}_{ME}^2$ denotes the estimated value of $\tau^2$ based on the mixed-effects model (i.e., the residual amount of heterogeneity). It can happen that $\hat{\tau}_{RE}^2 < \hat{\tau}_{ME}^2$, in which case $R^2$ is set to zero. Again, the value of $R^2$ will change depending on the estimator of $\tau^2$ used. Also note that this statistic is only computed when the mixed-effects model includes an intercept (so that the random-effects model is clearly nested within the mixed-effects model). You can also use the ''anova.rma.uni()'' function to compute $R^2$ for any two models that are known to be nested.+!!! The pseudo $R^2$ statistic (Raudenbush, 2009) is computed with $$R^2 = \frac{\hat{\tau}_{RE}^2 - \hat{\tau}_{ME}^2}{\hat{\tau}_{RE}^2},$$ where $\hat{\tau}_{RE}^2$ denotes the estimated value of $\tau^2$ based on the random-effects model (i.e., the total amount of heterogeneity) and $\hat{\tau}_{ME}^2$ denotes the estimated value of $\tau^2$ based on the mixed-effects model (i.e., the residual amount of heterogeneity). It can happen that $\hat{\tau}_{RE}^2 < \hat{\tau}_{ME}^2$, in which case $R^2$ is set to zero. Again, the value of $R^2$ will change depending on the estimator of $\tau^2$ used. Also note that this statistic is only computed when the mixed-effects model includes an intercept (so that the random-effects model is clearly nested within the mixed-effects model). You can also use the ''[[https://wviechtb.github.io/metafor/reference/anova.rma.html|anova()]]'' function to compute $R^2$ for any two models that are known to be nested.
  
 ??? For random-effects models fitted with the rma() function, how is the prediction interval computed by the predict() function? ??? For random-effects models fitted with the rma() function, how is the prediction interval computed by the predict() function?
  
-!!! By default, the interval is computed with $$\hat{\mu} \pm z_{1-\alpha/2} \sqrt{\mbox{SE}[\hat{\mu}]^2 + \hat{\tau}^2},$$ where $\hat{\mu}$ is the estimated average true outcome, $z_{1-\alpha/2}$ is the $100 \times (1-\alpha/2)$th percentile of a standard normal distribution (e.g., $1.96$ for $\alpha = .05$), $\mbox{SE}[\hat{\mu}]$ is the standard error of $\hat{\mu}$, and $\hat{\tau}^2$ is the estimated amount of heterogeneity (i.e., the variance in the true outcomes across studies). If the model was fitted with the Knapp and Hartung (2003) method (i.e., with ''test="knha"'' in ''rma()''), then instead of $z_{1-\alpha/2}$, the $100 \times (1-\alpha/2)$th percentile of a t-distribution with $k-1$ degrees of freedom is used.+!!! By default, the interval is computed with $$\hat{\mu} \pm z_{1-\alpha/2} \sqrt{\mbox{SE}[\hat{\mu}]^2 + \hat{\tau}^2},$$ where $\hat{\mu}$ is the estimated average true outcome, $z_{1-\alpha/2}$ is the $100 \times (1-\alpha/2)$th percentile of a standard normal distribution (e.g., $1.96$ for $\alpha = .05$), $\mbox{SE}[\hat{\mu}]$ is the standard error of $\hat{\mu}$, and $\hat{\tau}^2$ is the estimated amount of heterogeneity (i.e., the variance in the true outcomes across studies). If the model was fitted with the Knapp and Hartung (2003) method (i.e., with ''test="knha"'' in ''[[https://wviechtb.github.io/metafor/reference/rma.uni.html|rma()]]''), then instead of $z_{1-\alpha/2}$, the $100 \times (1-\alpha/2)$th percentile of a t-distribution with $k-1$ degrees of freedom is used.
  
 Note that this differs slightly from Riley et al. (2001), who suggest to use a t-distribution with $k-2$ degrees of freedom for constructing the interval. Neither a normal, nor a t-distribution with $k-1$ or $k-2$ degrees of freedom is correct; all of these are approximations. The computations in metafor are done in the way described above, so that the prediction interval is identical to the confidence interval for $\mu$ when $\hat{\tau}^2 = 0$, which could be argued is the logical thing that should happen. Note that this differs slightly from Riley et al. (2001), who suggest to use a t-distribution with $k-2$ degrees of freedom for constructing the interval. Neither a normal, nor a t-distribution with $k-1$ or $k-2$ degrees of freedom is correct; all of these are approximations. The computations in metafor are done in the way described above, so that the prediction interval is identical to the confidence interval for $\mu$ when $\hat{\tau}^2 = 0$, which could be argued is the logical thing that should happen.
Line 83: Line 83:
 ??? How is the Freeman-Tukey transformation of proportions and incidence rates computed? ??? How is the Freeman-Tukey transformation of proportions and incidence rates computed?
  
-!!! The ''escalc()'' and ''rma()'' functions offer the possibility to transform raw proportions and incidence rates with the Freeman-Tukey transformation (Freeman & Tukey, 1950). For proportions, this is also sometimes called the 'Freeman-Tukey double arcsine transformation'.+!!! The ''[[https://wviechtb.github.io/metafor/reference/escalc.html|escalc()]]'' function offers the possibility to transform raw proportions and incidence rates with the Freeman-Tukey transformation (Freeman & Tukey, 1950). For proportions, this is also sometimes called the 'Freeman-Tukey double arcsine transformation'.
  
 For proportions, the transformation (''measure="PFT"'') is computed with the equation $y_i = 1/2 \times (\mbox{asin}(\sqrt{x_i/(n_i+1)}) + \mbox{asin}(\sqrt{(x_i+1)/(n_i+1)}))$, where $x_i$ denotes the number of individuals experiencing the event of interest and $n_i$ denotes the total number of individuals (i.e., sample size). The variance of $y_i$ is then computed with $v_i = 1/(4n_i + 2)$. For proportions, the transformation (''measure="PFT"'') is computed with the equation $y_i = 1/2 \times (\mbox{asin}(\sqrt{x_i/(n_i+1)}) + \mbox{asin}(\sqrt{(x_i+1)/(n_i+1)}))$, where $x_i$ denotes the number of individuals experiencing the event of interest and $n_i$ denotes the total number of individuals (i.e., sample size). The variance of $y_i$ is then computed with $v_i = 1/(4n_i + 2)$.
Line 93: Line 93:
 ??? Why do I get different results with the Mantel-Haenszel method as implemented in metafor when compared to other software? ??? Why do I get different results with the Mantel-Haenszel method as implemented in metafor when compared to other software?
  
-!!! When used with the default settings, the ''rma.mh()'' function in metafor can indeed provide results that differ from those obtained with other meta-analytic software, such as the ''metan'' function in Stata, the Review Manager (RevMan) from the Cochrane Collaboration, or Comprehensive Meta-Analysis (CMA). By default, metafor does not apply any adjustments to the cell counts in studies with zero cases in either group when applying the Mantel-Haenszel method, while other software may do this automatically. For more details, take a look at the [[tips:comp_mh_different_software|comparison of the Mantel-Haenszel method in different software]] and what settings to use to make metafor provide the exact same results as other software.+!!! When used with the default settings, the ''[[https://wviechtb.github.io/metafor/reference/rma.mh.html|rma.mh()]]'' function in metafor can indeed provide results that differ from those obtained with other meta-analytic software, such as the ''metan'' function in Stata, the Review Manager (RevMan) from the Cochrane Collaboration, or Comprehensive Meta-Analysis (CMA). By default, metafor does not apply any adjustments to the cell counts in studies with zero cases in either group when applying the Mantel-Haenszel method, while other software may do this automatically. For more details, take a look at the [[tips:comp_mh_different_software|comparison of the Mantel-Haenszel method in different software]] and what settings to use to make metafor provide the exact same results as other software.
  
 ==== References ==== ==== References ====
faq.txt · Last modified: 2023/01/24 07:56 by Wolfgang Viechtbauer