faq
Differences
This shows you the differences between two versions of the page.
Previous revision | |||
— | faq [2023/01/24 07:56] (current) – [General Questions] Wolfgang Viechtbauer | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ===== Frequently Asked Questions ===== | ||
+ | |||
+ | ~~FAQ~~ | ||
+ | |||
+ | =?== General Questions ==== | ||
+ | |||
+ | ??? Why is the package called ' | ||
+ | |||
+ | !!! The name ' | ||
+ | |||
+ | ??? What is (was) the ' | ||
+ | |||
+ | !!! Based on some code I wrote as part of my dissertation research, I developed a function called '' | ||
+ | |||
+ | ??? Is the package validated in some way? | ||
+ | |||
+ | !!! Various attempts have been made to validate the functions in the metafor package. First of all, when corresponding analyses could be carried out, I have compared the results provided by the metafor package with those obtained with other software packages for several data sets. In particular, results have been compared with those provided by the '' | ||
+ | |||
+ | Second, results provided by the metafor package have been compared with published results described in articles and books (the assumption being that those results are in fact correct). On this website, I provide a number of such [[analyses|analysis examples]] that you can examine yourself. All of these examples (and some more) are also encapsulated in automated tests using the [[https:// | ||
+ | |||
+ | Third, I have conducted extensive simulation studies for many of the methods implemented in the package to ensure that their statistical properties are as one would expect based on the underlying theory. To give a simple example, under the assumptions of an equal-effects model (i.e., homogeneous true effects, normally distributed effect size estimates, known sampling variances), the empirical rejection rate of $H_0: \theta = 0$ must be nominal (within the margin of error one would expect when randomly simulating such data). This is in fact the case, providing support that the '' | ||
+ | <code rsplus> | ||
+ | library(metafor) | ||
+ | k <- 5 # number of studies to simulate in each iteration | ||
+ | pval <- rep(NA, 10000) | ||
+ | for (i in 1:10000){ | ||
+ | vi <- runif(k, .01, 1) # simulate sampling variances | ||
+ | yi <- rnorm(k, 0, sqrt(vi)) | ||
+ | | ||
+ | } | ||
+ | mean(pval <= .05) # compute empirical rejection rate (should be ~0.05) | ||
+ | </ | ||
+ | Similar (and much more thorough/ | ||
+ | |||
+ | It may also be useful to note that there is now an appreciable user base of the metafor package. The [[https:// | ||
+ | |||
+ | Finally, I have become very proficient at hitting the [[https:// | ||
+ | |||
+ | ??? Is the package development funded? | ||
+ | |||
+ | !!! For the most part, the development of the package has been funded through my own precious time. Through some collaborative work on the 'Open Meta-Analyst' | ||
+ | |||
+ | ??? How do I cite the package? | ||
+ | |||
+ | !!! First of all, thanks for trying to do so in the first place. The best way of citing the package is to cite the following paper: | ||
+ | |||
+ | Viechtbauer, | ||
+ | |||
+ | By the way, try '' | ||
+ | |||
+ | ??? Are there other R packages for meta-analysis? | ||
+ | |||
+ | !!! There are actually many R packages available for conducting meta-analyses. To get an appreciation for what the " | ||
+ | |||
+ | ??? Why can I not just use the lm(), lme(), and lmer() functions to conduct my meta-analysis? | ||
+ | |||
+ | !!! First of all, meta-analytic models (as can be fitted with the '' | ||
+ | |||
+ | Furthermore, | ||
+ | |||
+ | =?== Technical Questions ==== | ||
+ | |||
+ | ??? How are $I^2$ and $H^2$ computed in the metafor package? | ||
+ | |||
+ | !!! For random-effects models, the $I^2$ statistic is computed with $$I^2 = 100\% \times \frac{\hat{\tau}^2}{\hat{\tau}^2 + \tilde{v}}, | ||
+ | |||
+ | Therefore, depending on the estimator of $\tau^2$ used, the values of $I^2$ and $H^2$ will change. For random-effects models, $I^2$ and $H^2$ are often computed in practice with $I^2 = 100\% \times (Q-(k-1))/ | ||
+ | |||
+ | These two sets of equations for $I^2$ and $H^2$ actually coincide when using the DerSimonian-Laird estimator of $\tau^2$ (i.e., the commonly used equations are actually special cases of the more general definitions given above). Therefore, if you prefer the more conventional definitions of these statistics, use '' | ||
+ | |||
+ | See the analysis example for [[analyses: | ||
+ | |||
+ | ??? For mixed-effects models, how is the $R^2$ statistic computed by the rma() function? | ||
+ | |||
+ | !!! The pseudo $R^2$ statistic (Raudenbush, | ||
+ | |||
+ | ??? For random-effects models fitted with the rma() function, how is the prediction interval computed by the predict() function? | ||
+ | |||
+ | !!! By default, the interval is computed with $$\hat{\mu} \pm z_{1-\alpha/ | ||
+ | |||
+ | Note that this differs slightly from Riley et al. (2011), who suggest to use a t-distribution with $k-2$ degrees of freedom for constructing the interval. Neither a normal, nor a t-distribution with $k-1$ or $k-2$ degrees of freedom is correct; all of these are approximations. The computations in metafor are done in the way described above, so that the prediction interval is identical to the confidence interval for $\mu$ when $\hat{\tau}^2 = 0$, which could be argued is the logical thing that should happen. If the prediction interval should be computed exactly as described by Riley et al. (2011), one can use argument '' | ||
+ | |||
+ | ??? How is the Freeman-Tukey transformation of proportions and incidence rates computed? | ||
+ | |||
+ | !!! The '' | ||
+ | |||
+ | For proportions, | ||
+ | |||
+ | For incidence rates, the transformation ('' | ||
+ | |||
+ | One can also find definitions of these transformations without the multiplicative constant $1/2$ (the equations for the variance should then be multiplied by $4$). Since the $1/2$ is just a constant, it does not matter which definition one uses (as long as one uses the correct equation for the sampling variance). The metafor package uses the definitions given above, so that values obtained from the arcsine square-root (angular) transformation ('' | ||
+ | |||
+ | ??? Why do I get different results with the Mantel-Haenszel method as implemented in metafor when compared to other software? | ||
+ | |||
+ | !!! When used with the default settings, the '' | ||
+ | |||
+ | ==== References ==== | ||
+ | |||
+ | Freeman, M. F., & Tukey, J. W. (1950). Transformations related to the angular and the square root. //Annals of Mathematical Statistics, 21//(4), 607--611. https:// | ||
+ | |||
+ | Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. // | ||
+ | |||
+ | van Houwelingen, | ||
+ | |||
+ | Lipsey, M. W., & Wilson, D. B. (2001). //Practical meta-Analysis.// | ||
+ | |||
+ | Raudenbush, S. W. (2009). Analyzing effect sizes: Random effects models. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), //The handbook of research synthesis and meta-analysis// | ||
+ | |||
+ | Riley, R. D., Higgins, J. P. T. & Deeks, J. J. (2011). Interpretation of random effects meta-analyses. //British Medical Journal, 342//, d549. https:// | ||
+ | |||
+ | Sterne, J. A. C. (Ed.) (2009). // | ||