tips:comp_mh_different_software
Differences
This shows you the differences between two versions of the page.
Last revision | |||
— | tips:comp_mh_different_software [2019/05/05 16:21] – external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ===== Comparison of the Mantel-Haenszel Method in Different Software ===== | ||
+ | |||
+ | The Mantel-Haenszel method is an approach for fitting meta-analytic fixed-effects models when dealing with studies providing data in the form of 2x2 tables or in the form of event counts (i.e., person-time data) for two groups (Mantel & Haenszel, 1959). The method is particularly advantageous when aggregating a large number of studies with small sample sizes (the so-called sparse data or increasing strata case). | ||
+ | |||
+ | The method is available in the metafor package via the '' | ||
+ | |||
+ | ==== Data Preparation ==== | ||
+ | |||
+ | The data to be used for this example are stored in the dataset '' | ||
+ | <code rsplus> | ||
+ | library(metafor) | ||
+ | dat <- dat.nielweise2007 | ||
+ | dat | ||
+ | </ | ||
+ | <code output> | ||
+ | | ||
+ | 1 1 Bach 1996 0 116 3 117 | ||
+ | 2 2 | ||
+ | 3 3 Maki 1997 2 208 9 195 | ||
+ | 4 4 Raad 1997 0 130 7 136 | ||
+ | 5 5 Heard 1998 5 151 6 157 | ||
+ | 6 6 | ||
+ | 7 7 | ||
+ | 8 8 Marik 1999 1 74 2 39 | ||
+ | 9 9 | ||
+ | 10 10 Sheng 2000 1 113 2 122 | ||
+ | 11 11 Chatzinikolaou 2003 0 66 7 64 | ||
+ | 12 12 | ||
+ | 13 13 | ||
+ | 14 14 Leon 2004 6 187 11 180 | ||
+ | 15 15 Yucel 2004 0 118 0 105 | ||
+ | 16 16 Moretti 2005 0 252 1 262 | ||
+ | 17 17 Rupp 2005 1 345 3 362 | ||
+ | 18 18 Osma 2006 4 64 1 69 | ||
+ | </ | ||
+ | Variables '' | ||
+ | |||
+ | ==== Mantel-Haenszel Method ==== | ||
+ | |||
+ | An analysis of these data using the Mantel-Haenszel method can be carried out with: | ||
+ | <code rsplus> | ||
+ | res1 <- rma.mh(measure=" | ||
+ | print(res1, digits=3) | ||
+ | </ | ||
+ | <code output> | ||
+ | Fixed-Effects Model (k = 18) | ||
+ | |||
+ | Test for Heterogeneity: | ||
+ | Q(df = 16) = 16.864, p-val = 0.394 | ||
+ | |||
+ | Model Results (log scale): | ||
+ | |||
+ | estimate | ||
+ | -1.209 | ||
+ | |||
+ | Model Results (OR scale): | ||
+ | |||
+ | estimate | ||
+ | | ||
+ | |||
+ | Cochran-Mantel-Haenszel Test: CMH = 32.214, df = 1, p-val < .001 | ||
+ | Tarone' | ||
+ | </ | ||
+ | Therefore, the odds ratio is estimated to be .299 (with 95% CI: 0.193 to 0.462). In other words, the odds of an infection are estimated to be approximately 70% lower (i.e., $(1 - .299) \times 100%$) in patients receiving an anti-infective-treated catheter instead of a standard catheter. The overall effect is clearly statistically significant (with both the Wald-type z-test and the Cochran-Mantel-Haenszel chi-square test in close agreement). The Q-test for heterogeneity is not significant ($Q(16) = 16.86, p = .39$), although Tarone' | ||
+ | |||
+ | ==== Results from Stata ==== | ||
+ | |||
+ | The same analysis run in Stata using the '' | ||
+ | <code output> | ||
+ | | ||
+ | ---------------------+--------------------------------------------------- | ||
+ | Bach (1996) | ||
+ | Brun-Buisson (2004) | ||
+ | Chatzinikolaou (2003 | 0.058 | ||
+ | Collin (1999) | ||
+ | Corral (2003) | ||
+ | George (1997) | ||
+ | Hannan (1999) | ||
+ | Heard (1998) | ||
+ | Leon (2004) | ||
+ | Maki (1997) | ||
+ | Marik (1999) | ||
+ | Moretti (2005) | ||
+ | Osma (2006) | ||
+ | Pierce (2000) | ||
+ | Raad (1997) | ||
+ | Rupp (2005) | ||
+ | Sheng (2000) | ||
+ | Yucel (2004) | ||
+ | ---------------------+--------------------------------------------------- | ||
+ | M-H pooled OR | 0.317 | ||
+ | ---------------------+--------------------------------------------------- | ||
+ | |||
+ | Heterogeneity chi-squared = 16.41 (d.f. = 16) p = 0.425 | ||
+ | I-squared (variation in OR attributable to heterogeneity) = 2.5% | ||
+ | |||
+ | Test of OR=1 : z= 5.36 p = 0.000 | ||
+ | </ | ||
+ | The following forest plot is also generated: | ||
+ | |||
+ | {{ tips: | ||
+ | |||
+ | Note that the estimated overall odds ratio (and corresponding CI) is slightly different than the one obtained earlier. Also, the z-test of the overall effect and the chi-square test for heterogeneity are slightly different. | ||
+ | |||
+ | ==== Results from RevMan ==== | ||
+ | |||
+ | After entering the same data into the Review Manager and running the analogous analysis yields the following results: | ||
+ | |||
+ | {{ tips: | ||
+ | |||
+ | These results match what is reported by Stata and are again slightly different compared to the results obtained with metafor. | ||
+ | |||
+ | ==== Results from CMA ==== | ||
+ | |||
+ | Finally, the figure below shows the results from Comprehensive Meta-Analysis (CMA). These results match those obtained with Stata and RevMan and differ slightly from those obtained with metafor. | ||
+ | |||
+ | {{ tips: | ||
+ | |||
+ | ==== Reason for the Difference ==== | ||
+ | |||
+ | The results differ because studies with zero cases in either group are handled by default in a different way in metafor compared to Stata, RevMan, and CMA. To understand this better, note that the Mantel-Haenszel method itself does not require the calculation of the observed outcomes of the individual studies (in the present example, the observed (log) odds ratios of the $k$ studies) and instead directly makes use of the 2×2 table counts. Zero cells are not a problem (except in some extreme cases, such as when there are zero cases in one or both groups across all of the 2×2 tables). Therefore, it is unnecessary to add some constant to the cell counts of a study with zero cases in either group. However, both Stata, RevMan, and CMA apply an adjustment (often called a continuity correction) to the cell counts in such studies (but studies with zero cases in both groups are dropped/ | ||
+ | |||
+ | By default, metafor uses this adjustment when calculating the observed outcomes (the observed log odds ratios) of the $k$ studies (here, zero cells can be problematic, | ||
+ | |||
+ | We can, however, adjust the settings, so that metafor also applies the cell count adjustment, not only when calculating the observed outcomes, but also when carrying out the computations of the Mantel-Haenszel method. For this, we have to adjust the defaults of the '' | ||
+ | <code rsplus> | ||
+ | res2 <- rma.mh(measure=" | ||
+ | print(res2, digits=3) | ||
+ | </ | ||
+ | <code output> | ||
+ | Fixed-Effects Model (k = 17) | ||
+ | |||
+ | Test for Heterogeneity: | ||
+ | Q(df = 16) = 16.406, p-val = 0.425 | ||
+ | |||
+ | Model Results (log scale): | ||
+ | |||
+ | estimate | ||
+ | -1.149 | ||
+ | |||
+ | Model Results (OR scale): | ||
+ | |||
+ | estimate | ||
+ | | ||
+ | |||
+ | Cochran-Mantel-Haenszel Test: CMH = 30.919, df = 1, p-val < .001 | ||
+ | Tarone' | ||
+ | </ | ||
+ | |||
+ | These are the exact same results as obtained with Stata, RevMan, and CMA. However, the results of Bradburn et al. (2007) suggest that the '' | ||
+ | |||
+ | ==== References ==== | ||
+ | |||
+ | Bradburn, M. J., Deeks, J. J., Berlin, J. A., & Localio, A. R. (2007). Much ado about nothing: A comparison of the performance of meta-analytical methods with rare events. // | ||
+ | |||
+ | Harris, R. J., Bradburn, M. J., Deeks, J. J., Harbord, R. M., Altman, D. G., & Sterne, J. A. C. (2008). metan: Fixed- and random-effects meta-analysis. //The Stata Journal, 8//(1), 3--28. URL: http:// | ||
+ | |||
+ | Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. //Journal of the National Cancer Institute, 22//(4), 719--748. | ||
+ | |||
+ | Niel-Weise, B. S., Stijnen, T., & van den Broek, P. J. (2007). Anti-infective-treated central venous catheters: A systematic review of randomized controlled trials. //Intensive Care Medicine, 33//(12), 2058--2068. | ||
+ | |||
+ | Review Manager (RevMan) [Computer program] (Version 5.3). Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, | ||
+ | |||
+ | Sterne, J. A. C. (Ed.) (2009). // | ||
+ | |||
+ | Sweeting, M. J., Sutton, A. J., & Lambert, P. C. (2004). What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. // | ||
tips/comp_mh_different_software.txt · Last modified: 2021/11/08 15:48 by Wolfgang Viechtbauer